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ABSTRACT 

This handbook describes the processes used in a 
project to identify technological changes taking placd in business 
and industry and to infuse content related to these changes into the 
curricula of tecnnical colleges and local education agencies in rural 
areas. The handbook includes 15 modules. The first module explains 
the system developed by the project, and the other modules present 
more specific processes. Topics covered are the following: 
high-technology curriculum updating processes; envisioning process; 
needs assessment; planning and conducting statewide or regional 
surveys and interview surveys of businesses and industries; planning 
and conducting follow-up studies; planning and conducting a Delphi 
-.tufly; selecting appropriate sampling techniques and determining 
valid sample sizes; working with small groups; necessary skills 
matrix; articulation — helping students transition from secondary to 
postsecondary institutions; module development guidelines; 
evaluation? work-based learning experiences; and developing 
partnerships between education and business and industry. (KC) 
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PREFACE 

This handbook describes the processes project staff members 
and participating teachers used to identify technological changes 
taking place in business and industry and to infuse content 
related to these changes in their curriculums. The primary 
purposes of the High-Technology Training Model Project were to 
(1) develop a practical method to monitor technological changes 
in B/I and, (2) design a process to up-date vocational, 
technical, and technology education curriculums. 

This handbook includes fifteen modules. Their topics range 
from articulation to working with small groups. Some focus on a 
specific process and others are more general. We recommend that 
you read the first module, "High-Technology Curriculum Up-Dating 
Processes, 11 to gain an understanding of the system developed by 
the project. The other modules present more specific processes 
and can be read or used individually* Where there are ties to 
other modules, these are noted and the other module (s) is 
referenced. 

The project officially ended on September 30, 1990. 
However, development of the system and processes described in 
this handbook continues. New opportunities to apply these 
modules continue to arise in our work. If you have questions or 
comments, please share them with us. We can be reached by phone 
at (715) 232-1382. 

Orville Nelson 
Tim Mero 

* * 
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High-Techn logy Curriculum Updating Process 

Intr oduction 

This module describes tf. processes used in the High- 
Technology Training Model Project to identify and develop new 
content for high school and postsecondary courses. Techniques 
for identifying new technology being used by local business and 
industry, procedures for isolating the competencies related to 
this technology, and processes for developing instructional 
modules are included. This module presents a discussion of the 
system used. In soue instances the reader will be referred to 
other modules for more detailed instructions on how to conduct 
certain activities. 

Purpose 

The purpose of this module is to provide a comprehensive 
description of the processes used to identify new competencies 
and update courses. 

Objectives 
This module is designed to: 

1. Describe the system used to identify high-technology 
competencies and revise courses.* 

2. Provide the information and procedures needed to plan and 
conduct the high-technology curriculum enhancement process . 

3. Provide the instructions and procedures needed to identify 
changes in technology and related systems in the workplace . 

4. Define a process for analyzing the change data to identify 
relevant content for existing programs and courses. 



5. Describe how the process can be used to update courses. 

Applications of the Proces s 

The processes described in this module can be useful in 
situations where it is important to keep programs and courses 
current with the technology being used in the workplace. It can 
also be used to identify applications of more traditional content 
in math, science and communications. The process was designed to 
identify new content that can be infused in existing courses. 
The intent was not to develop new programs per say. However, 
there may be situations in which the content has changed so 
drastically, that it would merit developing a new course or 
courses. When this happens one should also look at what can be 
eliminated from the curriculum. 

The instructional design portion of the model is based on a 
modulerized approach to instructional development and competency- 
based education. The module approach was used to provide 
flexibility. A module is designed around a competency or related 
set of competencies. They are designed to be used and linked in 
a variety of conf iguratioms to meet a variety of needs. For 
example, a set of modules could be configured to provide the 
basis for a course at the technical college level. A sub-set of 
these modules could be selected and used to provide specific in- 
plant training for a company. The modules are not designed to be 
used by students as an individualized package. A knowledgeable 
instructor plus the module are required to produce effective 
learning. 
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The ir.odel also uses a team approach to identifying the 
competencies and developing the modules. Ideally, teachers from 
a variety of disciplines in a school district should be involved 
in the business/industry interviews. This will allow them to 
determine how their disciplines relate to what is happening in 
the world of work. It will also provide them with ideas and 
problems for activities in their classroc is and laboratories. 
When possible, the process should include teachers from one and 
two year postsecondary programs. These teachers also need to 
keep current with technological changes, moreover, their 
technical e cpertise will contribute an additional dimension to 
the curriculum development workshops. 

The Process 

The flow chart in Figure 1 describes the sequence of steps 
that was involved in the high-technology curriculum development 
process. Each of the steps is described in more detail in the 
paragraphs that follow, in addition, some of the steps have more 
detailed instructions presented in separate modules. This 
approach has been successfully used. However, you may find it 
necessary to tailor it to your particular situation. Use it as a 
guide and modify it as necessary. 
Orientation 

The first portion of this process focuses on the 
identification of new technology that is being used in the 
workplace or is being considered by local companies, one of the 
major activities is a series of interviev/s in business and 

• « 
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industry. in order to maximize the effectiveness of these 
interviews, educators need to know what information is needed and 
how it will be used in later stages in the process. Therefore, 
it is necessary during the orientation to orient them to major 
changes that have been taking place in the area labor market, the 
business/industry interview process, and the nature of 
competency-based education. 
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Figure 1 High-Tech Curriculum Updating Process 
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The amount of time required for this orientation will depend 
upon the experience and background of the teachers involved, if 
they are experienced in developing competency based instructional 
materials, this portion of the orientation will take only a brief 
period of time. If they are not, they will need an orientation 
to the basic principles. An outline of the topics for the 
orientation session is given in Figure 2. 

Fi qure 2. Outline for Participating Educator Orientation 

1. Importance of updating courses 

2. Competency Based Education (CBE) 

a. Nature of competencies 

b. Identifying competencies 

c. Competency based objectives 

d. Designing CBE learning experiences 

e. Competency based evaluation 

3. Major changes in the labor market 

a. Labor market data 

b. Follow-up studies 

c Changes in the past decade 
d. Major trends 

4. High-Tech Curriculum Development Process 

a. Philosophy 

b. steps 

c. Procedures 

(Figure cont,) 
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5. Businesses/Industry Interviews 

a. Purpose 

b. Process 

c. Interviewing techniques 

d. Interview questions 

e. Form interview teams 

f. Select B/I to visit 

g. Procedures for processing the results 
B/I Interview Surveys 

The purpose of the B/I interview surveys is to have 
educators interact with people from area companies in a semi- 
structured setting. Educators have questions to ask but also 
have the freedom to discuss other topics. The questions will 
direct them to gain information on technology that has been 
recently added to the workplace and new technologies being 
considered. In addition, teachers need to look for applications 
of their disciplines. 

The first step in the business interview process is for the 
participating teachers to select the companies they will visit. 
If several people are interested in visiting the same company, 
they should consider teaming up and going on the same visit. 
This would save time for the people being interviewed at the 
company. Also, it would allow the teachers to compare notes on 
their visits and interviews. The companies selected should give a 
cross-section of the types of employment opportunities available 
in the cor.minity and area. In addition, some of the companies 
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should be innovative, i.e., those who are leaders in adopting new 
technology. 

The module on the business/industry interviews presents more 
detailed instructions on how to conduct the interviews, questions 
asked, and the procedures t > be employed in setting up the 
interviews. A copy of the interview questions is included in 
Appendix A of this module. 
Follow-up Studies 

Educators participating in the project were asked to conduct 
follow-up studies with former students who were one and five 
years out of high school. The one-year group was selected to 
provide information on the transition from school to work or 
advanced education. The five-year group was included to provide 
information on the post-secondary education and work experiences 
of former students. 

A sample set of follow-up survey forms is included in 
Appendix B. These surveys provide valuable feedback on the 
extent to which high school courses, programs and experiences 
have prepared students for the tasks and situations they face 
after high school. The open-ended questions at the end of the 
survey provide information on the new technology employers have 
implemented and the types of changes former students are 
experiencing in their work. 

A specific module on follow-up studies has been developed. 
The reader should refer to this when planning and conducting the 
follow-up study. Modules on follow-up studies and sampling are 
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included in this handbook. 

In many instances, schools follow-up with their graduates. 
However, it is also recommended that students who do not graduate 
be contacted to find out what their educational and work 
experiences have been since leaving school. These former 
students may be able to offer some valuable suggestions cn how 
programs could be improved in order to retain more students 
through graduation. 

The follow-up studies also provide information on the former 
students' employers. When students provide employers' names and 
addresses, it is possible to contact employers and obtain 
feedback on graduates. The purpose of this type of employer 
follow-up survey is not to evaluate specific students but to 
provide a reference point to use in evaluating the degree to 
which your school's programs have developed the competencies and 
skills needed by successful employees. 

General surveys of employers and business/ industry 
interviews described in Step 2 give feedback on the employers' 
perceptions of the entering competencies of the general work 
force. Often times there is a halo affect in these perceptions. 
The traits of a few poorly qualified employees may be generalized 
to all employees. Therefore, it is important to obtain feedback 
on former students who have completed specific types of programs 
in your school system. Sample employer follow-up survey forms 
are included in Appendix C. Also, the module on follow-up 
studies will be useful in conducting this type of follow-up 
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study. 

To be effective, follow-up studies should be conducted on a 
regular basis „ It may not be necessary to conduct a five-year 
follow-up study with every group of high school students* 
However, a systematic plan and schedule should be developed for 
doing these. It would be more important to conduct a one-year 
follow-up study each year. This follow-up study is very 
sensitive to the types of changes that are taking place in the 
workplace and the needs of students who are leaving school for 
education or employment. 

An effective follow-up study will require at least two to 
three months to complete. Thus, it will need to be planned and 
scheduled so that the results are available for later steps in 
the curriculum development process. 
Monitor Area Labor Market Data 

Information on sources of labor market data was included in 
the orientation session. This session also provided some up-to- 
date information on area labor market trends. Teachers need to 
be encouraged to monitor these information sources, area 
newspapers, television newscasts, and other area events to 
identify changes taking place in the regional labor market. In 
addition, one or two staff people should be identified and asked 
to give special attention to monitoring area labor market data. 
These people might regularly review the regional and state 
publications on the labor market. In addition, they need to 
monitor local and area newspapers, journals, and business news on 
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TV. Since this information is not always of great interest to 
teachers in general, it would be well to select staff members who 
have a special interest in this type of information and/or need 
to regularly make use of it in their work. 

Other sources can provide effective input. An advisory 
committee comprised of people from business and industry will 
provide valuable input on changes in systems, technology, and 
works. Advisory committee members also should be asked to look 
ahead - at least to the next three to five years. A Delphi 
study could be conducted to obtain more long-term future 
projections. In addition, area labor market studies might be 
done by selected classes. 
Analyze/Synthesize Data 

The data acquired through the business/ industry interview 
surveys will not be readily quantifiable. Each interviewer will 
need to record the technology, ideas, and suggestions identified 
during each interview. The interviewers may want to do some 
summary tallies to determine the frequency with which certain 
types of new technology were mentioned during the interviews or 
observed during the visits. However, a lot of the information 
gathered during these interviews will be in the form of 
impressions and feelings developed by the interviewers. 

It is suggested that the educators participating in the 
business interview process meet in small groups and use a nominal 
group or similar process to identify the changes and competency 
needs they perceived during their interview visits. During the 
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initial part of this process, these changes and needs should be 
listed through a brainstorming process. After this has been 
done, these items can be grouped and prioritized. A module on 
small group processes is included in this handbook. 

After the technological changes have been identified, the 
groups can look at the implications of these changes for school 
programs. The types of competencies needed by people entering 
the work force can be identified. These competencies can then be 
related to specific courses and programs in your school. For 
example, the increasing use of CAD might be given a high priority 
during this group process. An analysis of the competencies 
required to use CAD might include the following. 

1. Keyboarding - math and business education courses. 

2. Drafting principles - technology education. 

3 . Style and design - art and technology education 
courses . 

4. Geometry used in designing and drafting - math 
courses. 

After the small groups have completed their discussions, 
they could meet and report their results to the total group. It 
may be desirable, at this time to redivide the groups by 
discipline or subject matter area and have them review the 
implications again. The competency lists and related courses 
identified by each group should be reviewed and consensus areas 
identified. For further .Information on group processes, review 
the module on this topic. 
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At this point, one also needs to consider the results of the 
follow-up study and the monitoring of area labor market data. 
The follow-up and labor market data will be in a more quantified 
form. There are various strategies that might be used to 
disseminate this information* Each staff member might be asked 
to read a report of the results of the follow-up study and a 
summary of area labor market trends* Considering human nature 
and the large number of documents typically distributed to 
educators, it would probably be more effective to have a team of 
two or three people responsible to analyze and report the results 
of the follow-up study and another team of similar size to report 
the results of monitoring area labor market data. It is 
recommended that the follow-up data be presented first. It deals 
more specifically with the products of your school programs and 
how they fare in labor market and advanced educational programs. 

The follow-up data should be presented and discussed with 
the participants in the small groups. The overall results should 
be presented first. After that has been done, the presenters may 
want to identify major outcomes they have perceived in their 
analyses. However, these conclusions need to be discussed by the 
teachers and administrators in the small groups. 

Next, the information on the local or area labor market 
should be presented. As with the follow-up results, one or two 
staff members could be responsible for synthesizing and 
presenting this information. In addition, they may want to 
develop a list of implications for your school 1 s programs and 
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courses. Alter these two presentations, the participating 
teachers and administrators can meet in their small groups to 
discuss the implications of this information. The same group 
procer< as used in stage one with the business/industry survey 
results :an be employed. Each group should develop a list of 
implications for your school programs and courses. 

After the implications have been identified, they should be 
contrasted with the results from the discussions of the 
business/industry interviews. Areas in which there is solid 
agreement reinforce each other and should be given more 
precedence in determining the priority for changes in courses and 
programs. Areas in which there are conflicts suggest a need for 
gathering more information to resolve these conflicts. 

The next step is to look at these implications ana isolate 
competencies that need to be developed. In some instances, the 
implications will be stated in a competency format, in other 
words, the types of behaviors and skills students need to have as 
they enter the work force or advanced educational programs will 
be embedded in the statements, if they are not, competency 
statements will have to be written which are specific enough so 
they can be compared with program and course objectives to 
determine if they are included or not. 

The next step will be to contrast the competency areas and 
competency statements developed by the small groups with the 
contents of specific courses. In order to do an effective job of 
this analysis, the course objectives need to be stated in 



behavioral or competency form. If they are not, it will be 
difficult to do effective analysis. During this analysis, two 
things need to be checked. First, you need to determine whether 
the competency is developed in existing courses. Second, if you 
find that current course content and activities relate to one or 
more of the competency areas identified, also check to be sure 
that students are learning how to apply these competencies to 
real world tasks. Often students are exposed to relevant content 
but there is a failure to relate these competencies to practical 
situations. As a result, students do not know how to make use of 
the knowledge they have as they work in practical situations. 

One effective way of conducting an analysis and determine 
the areas in which further work is needed, it to develop a course 
objective-competency matrix. A partial objective-competency 
matrix is given as an example in Figure 3 for the CAD 
illustration mentioned above. The rows of the matrix can be used 
to identify the objectives for a given course. The columns can 
be used to list the new competencies areas and competencies 
identified through the B/I interview, follow-up and labor market 
analysis processes. Checkmarks can be placed in the appropriate 
cells to indicate if and where the new competencies are covered. 
Another symbol can be used to identify potential areas for 
application of these competencies. 
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F igure 3, Objective - Competency Matrix 

Course: GRAPHIC COMMUNICATIONS I 

Computer Aided Design (CAD) 

Identified New Competencies 

Existing Course ~ "Drafting Principles Geometric 

Objective Keyboarding Principles of Design Analysis 

1. Use keyboard to 
center data X 

2 . Use mouse to 

enter data X 

3. 

• 

(n). 

Curriculum Development Workshop 

The next step in the process is to conduct a one week 
curriculum workshop for the project participants. The objectives 
of the workshop and its contents would be based on the needs of 
the participating educators. An example outline of a workshop 
agenda is given in Figure 4. Prior to the workshop, staff 
members should contact the participants and asked them to 
identify competency areas in which they wanted to work during the 
curriculum development phase. Their decisions should be based 
upon their interviews with local industry, local programs, and 
information that staff had provided. During the High-Technology 
Training Model Project, participating teachers identified four 
major areas in which they wanted to work: (a) new manufacturing 
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technology, (b) communications, (c) employability skills, and (d) 
self-managed work teams. In addition to these four areas, the 
project staff decided to include content on problem-solv.mg. 
Problem-solving had been identified as one of the major needs by 
companies in Wisconsin, thus, it was deemed important to include 
this. Since problem solving is a process, it could be woven into 
a variety of content modules. 

Select presenters who will do an effective job of providing 
content and technique sessions. Session on curriculum 
development techniques should focus on processes for developing 
instructional modules, writing behavioral objectives, and 
designing valid evaluation activities. 

If it is an objective of the workshop to "Vp participating 
educators develop a training module, then ~ icipants should be 
given adequate time to develop their individual modules as part 
of the workshop experience. F^r some participants, this may be 
their first formal, non-academic experience witn developing 
instructional material. Working with someone developing a 
similar module, or as a team in the development of a single 
module may relieve some of the associated anxiety. During the 
High-Technology Training Model Project Workshop, participating 
educators were encouraged to work as teams whenever possible, 
however, this did not occur in some instances. On the last day 
of the workshop for the High-Technology Training Model project, 
participants were asked to evaluate the workshop. Based on the 
participants evaluations, as well as the project staff's 
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experiences with the workshop format , it is recommended that an 
additional two days be added to make it a full week. This would 
provide adequate time for the participating educators to complete 
the writing of the first module. It is very important to 
complete the first module so that participants have an 
opportunity to get a feel for whe total process and also for the 
format of the module. 

The workshop format presents an environment within which 
people can share ideas and assist each other. It is important to 
have appropriate resource persons and reference materials 
available. As mentioned above it is important that the 
participants complete the first module before the end of the 
workshop. Writing additional modules is much easier. In fact, 
several of the educators participating in the High-Technology 
Training Model Projecc decided to write additional modules during 
the school year. 

Including postsecondary technical college and high school 
teachers in the same workshop, provides an opportunity to share 
ideas and information across educational levels. This can be 
beneficial and lead to some cooperative module development work. 
During the High-Technology Training Module Project f s workshop, 
some of the participants had three or four correlated modules 
available to them at the conclusion of the workshop. Each module 
was written by an individual participant; however, the linkages 
between modules were developed by the small group. 

Another factor in the success of the workshop was the set of 
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reinforcers or rewards set up for the participants. A monetary 
stipend was provided through the project budget. In some 
instances this stipend was supplemented by the participant's 
school system. In addition, college credit was available to the 
participants at a significantly reduced fee. In place of the 
usual fees of more than $100 per credit, the participant paid $10 
per credit. Also, if the person did not wish to enroll for 
credit, the workshop could be taken for Call's. Finally, a modest 
amount of money was available to each school for procuring audio- 
visual resources that supported the content of the module. 
Figure 4. Curriculum Workshop Agenda 

* Introduction to the high-technology curriculum 
revision process 

* High-technology skills - An industry perspective 

* Approaches to teaching thinking 

* Instructional module development 

* Small groups breakout sessions 

(Resource people discussed recent developments in 
three areas related to the participants' module 
development activities.) 

* Module development time 

* Work based learning experiences 

* Building fairness and address diversity 

* Articulation strategies and techniques 

* Evaluation techniques 

* Quality concepts (Figure cont.) 
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Module development time 

Module review - developer and a project staff 
member 



Write Instructional Module s 

As mentioned in the previous step, each participant started 
an instructional module during the High-Technology Training Model 
Project's curriculum development workshop. A standard format was 
presented to the participants. The rationale for this format and 
its specific contents were discussed in detail. A specific 
module has been developed on writing instructional modules. For 
more detailed information on the process, the reader should refer 
to the Module Development Guideline s included in this handbook. 

Two principles guided the design of the Instructional module 
format for educators participating in the High-Technology 
Training Model Project. First, the instructional module was to 
be competency-based. Second, it was to be designed to provide 
enough flexibility so the module could be used in different 
settings and linked with content in an existing course. Figure 5 
shows what the content of each module should include. 
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Figure jL. Instructional Module Content 
I. Title 
II. Description 
III. Name and School 
IV. Objectives 

A. Purpose 

B. Benefit of objectives 

C. Components of objectives 

1. Condition 

2. Performance 

3. Criteria 
V. Content Outline 

VI. Methodology 
VII. Activities 

A. Relationship to objectives 

B. Student 
VIII. Resources 

XI. Evaluation 

A. Relationship to objectives 

B. Paper and pencil 

C. Product/process 

During the High-Technology Training Model Project, workshop 
staff made it a point to work with each of the participating 
teachers and to contact them two or three times a day during the 
time they are working on their modules. Also, resource people in 
the areas of flexible manufacturing, communications, and small 
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group leadership were available. Emphasis was given to assuring 
that the content and learning activities directly related to the 
competencies identified in the objectives of the module. 
Participants had the most difficulty in developing an effective 
evaluation for their modules. Since the modules were competency- 
based and dealt with the use or application of high-technology 
concepts , the standard evaluation questions and techniques did 
not serve real well. One session in the workshop dealt with 
evaluation. However , there was not enough time to work with each 
participant and Apply the techniques to his/her module. If the 
workshop had been approximately two days longer, evaluation 
techniques could be discussed near the close of the workshop when 
the participants are ready to develop this portion of the module. 
This would be a more effective approach. Evaluation techniques 
that are appropriate to higher levels of the taxonomy; 
application, analysis and synthesis need to be presented. 
Try Out The Modules 

Instructors participating in the High-Technology Training 
Module Project were asked to try out their modules at appropriate 
points in their courses. Instructors from the postsecondary 
jechnicaJ college system are also 'encouraged to use the modules 
in special in-plant training as well as in their regular courses. 
Almost all of the participating teachers were able to try out 
their modules and have them evaluated by their students. 

During the try out, teachers were encouraged to identify 
areas that needed to be improved, expanded or changed. As these 
modifications were sent back to the project staff, the changes 
were entered into the projects 1 word processing system and 
revised copies were sent to the teacher. 
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Evaluate The Modules 

Three types of evaluation data were collected. The 
teacher's perceptions of the quality and effectiveness of the 
module were obtained through a teacher evaluation form. 
Students provided two types of evaluation information. First, 
their performance on the test developed by the author of the 
module was analyzed. Second, each student completed a rating 
scale on the module. The rating scale was developed by project 
staff so it was the same for each of the modules evaluated. 
All of the evaluation forms used are included at the end of this 
module in Appendix D. 

Overall, the teachers gave very positive evaluations to the 
modules they had developed and to the module format. This does 
not mean that they did not find areas in which the modules could 
be improved. Most of the teachers found something that they 
wanted to change and refine. However, they were satisfied with 
the module approach and indicated that it was a functional way to 
integrate new content in existing courses. 

Students also responded positively to the modules. They 
indicated that they had learned new content and competencies that 
would be useful to them in the "uture. They also reported that 
the modules were interesting and functional. More info tion on 
evaluation is given in the Evaluation Module included in nis 
handbook. 

Refine The Modules 

The most opportune time to refine a module is right after it 
has been used. At that point in time, the experiences gathered 
through using the module will be fresh in mind. Also, the 
student evaluation results will be readily available, and any 
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other comments and reactions that students may have had that were 
not recorded in the student feedback forms will be remembered. 

Student performance test results should be analyzed item by 
item to determine those areas in which more emphasis needs to be 
given. For example, if students perform poorly on the items 
related to one of the objectives in the module, the content and 
learning activities should be modified and/or expanded to assure 
that this objective is attained when the module is used again. 
After the module has been revised, the writer should look at the 
types of changes made and the implications for the module writing 
process. In other words, the process should be revised so that 
the same mistakes are not made in the future. The process of 
using and evaluating, the first module should provide feedback 
that would be valuable in making the following decisions. 

1. How much detail should one include in the content area? 

2. What types of class activities are most effective? 

3. What types of student assignments should be made? 

4. What types of activities are of most interest to 
students? 

5. What types of activities are most effective in 
accomplishing the module objectives? 

6. Were appropriate audio-visual aids included? 

7. What evaluation processes are most appropriate? 
Especially important here are the techniques that assess higher 
levels of learning such as application, analysis and synthesis 
(problem solving) . 

8. How efficiently does this module link with other 
content in the course? 

9. Is the amount of content covered in the module 



26 

3f; 



appropriate? 

The module writer may wish to make note of the types of 
changes that needed to be made and refer to these as additional 
modules are developed. The first few modules provide an 
excellent learning process if the evaluation phase is included 
and feedback provided. Also, the person who is responsible for 
the overall project can learn from looking at a summary of the 
types of changes that need to be made in the modules. This 
information can be used to improve the instruction on the module 
development process. 
Integrate Modules Into Courses 

Once a module or set of modules has proven to be effective 
and relevant, it should be integrated into the overall course. 
During the try-out and evaluation phases, the modules were used 
in an appropriate place within a course. After they have been 
used, tested and refined, it is appropriate to A y integrate 
them into the course outline. This can be done by identifying 
where they will be taught and writing the introductory material 
so that the module links together with the instruction the 
precedes and follows it. As more modules are developed the 
teacher can link these together to form units of instruction and 
full courses. Often it will be necessary to develop some 
instructions for the teachers and materials for students that 
will facilitate the transition from one module to another. 

As educators become more competent in developing and using 
modules, they should be able to select from a set of existing 
modules and quickly put together a unit of instruction or a 
specialized training program. Research, conducted by the 
University of Wisconsin-Stout's Center for vocational, Technical 
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and Aauit Education indicates that business and industry want to 
have training programs that are ready when they need them, high 
quality, and tailored to their needs. An easy way to remember 
this is to use the acronym Q 2 T. The acronym stands for quick, 
quality and tailored. Quick represents the idea that companies 
want to have training delivered as soon as possible after a 
specific need has been identified. They usually do not want to 
wait until a n^w semester or a new school term starts. Quality 
reflects the desire of business and industry to have the best 
inst± ctors and the best instructional materials that are 
available related to their needs. They are concerned that the 
instruction be effective and that their employees develop new 
work patterns which will be ruore productive. Tailored indicates 
that the content of a training program must be focused 
specifically on the needs of a particular business or industry. 
A company does not want to have time spent on the competencies 
and concepts that do net relate to its needs. Moreover, 
employees want competencies that they can use directly in their 
work. They will lose interest when content that is not directly 
related to, or does not have an obvious link with their training 
needs, is presented. This can cause them to lose interest and 
drop out of the training program. 

Assuming that a va? .d needs assessment has been done and 
that relevant modules have been selected, some fairly simple 
changes can be made in the modules to make them more interesting 
and relevant to the participants. If the module is valid in 
relation to the need, you will not want to change the 
competencies that are being developed by the module. In other 
words, the content is going to remain relatively stable. 
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However, the teaching examples and learning activities, i.e., the 
processes used in instruction, can be modi tied to make the 
modules more relevant. For example, if a training program on 
programmable logic controllers (PLCs) is taxight, the types of 
PLCs discussed can be those used in the plant and the problems 
given for students to solve could deal with the types of 
applications found in the company. 

An effective process to use in identifying these 
applications and examples is to do a pre-training program visit 
to the company. During this visit, you will be able to obtain a 
better idea of the specific needs of the people who will be in 
the training program and their current level of expertise. In 
addition, information on the types of equipment used, problems 
encountered, and applications can be acquired. This will allow 
you to tailor the instruction to the needs, background and 
interests of the employees to be involved in the training 
program. 

Continue to Monitor Technological Change 

There is an old phrase that "there are only two things that 
are certain in life....". Another item can be added to that 
list, change. As new knowledge is developed at an increasing 
pace and the world marketplace develops, change is going to be a 
constant in our lives. 

The high technology curriculum revision process was designed 
to be a dynamic process. One of the important steps in making 
this process dynamic is the continued monitoring of change. 
Steps two, three and four in the process provide valuable data on 
changes taking place in the workplace. These steps need to be 
continued. Staff members should be involved in visiting and 
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interviewing people in business and industry on a regular basis. 
This activity might be scheduled each calendar year or it could 
be done every other year. The two-year cycle, however, is the 
longest that should be maintained between these visits. In fact, 
it would be better to have a continuing relationship with the 
companies in your labor market area. 

Follow-up survey information from former students and 
graduates will provide an ongoing picture of changes in the 
workplace. One-year follow-up surveys identify problems that 
former students have in making transitions into the workplace and 
advanced educational programs. The five-year follow-up studies 
provide information on changing technology in the workplace and 
the needs of former students as they advance in their careers. 
Input from employers should also be obtained as a part of the 
follow-up process. 

Data on the area labor market needs to be continually 
monitored. A minimum of one or two staff members should be 
assigned to monitoring and assembling this information. Special 
reports provide valid information on the changing nature of your 
area labor market. Feedback from graduates and former students 
also helps to identify the appropriate labor market region for 
your school district and the types of changes taking place within 
it. other sources of information include scanning professional 
literature, participating in conferences and attending trade 
shows. Students who are working on cooperative education 
assignments or other work based learning experiences should be 
asked to identify the types of equipment and systems they are 
working with and the changes they see taking place at their job 
site. Also, the staff members who supervise these students out 
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at the worksite should be alert to new competencies required and 
changes in the technology being used by the employers. 

Advisory committees can provide valuable input on trends in 
the area labor market, new technology and systems being 
implemented in the workplace, and the types of new technology 
being considered for future implementation. Typically, each 
vocational program will have an occupational advisory committee. 
At least a portion of one meeting per year should be devoted to 
discussing "ew technology, new competencies needed and future 
trends in technology and work systems. The advisory committee 
can also be a valuable sounding board and provide assistance in 
determine the new competencies that should be placed in courses 
in your school programs. The asults of the small group staff 
sessions should be presented to the advisory committee for its 
reactions and suggestions. 

Finally, the effectiveness of the modules developed should 
be evaluated. The results of these evaluation activities must be 
used to improve the modules and also to improve the module 
development process. 
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APPENDIX A 
Business/Industry Interview Questions 
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II Hill TlXUNOUHiY TI<A 1 N I NO HODKL I'ttOJI-XT 



L'VTAK 



I) i root ions : Inlerview three to five people in local bus inosses or industries. 
II possible, select H/i's that employ your gradua tes . Record the major 
communis on this form. You may want to ask other questions as well. 

Bu s iness/ 1 nd u s try , . Da t e 

Ad dross City Zip 

Person (s ) Interviewed : _ 

Interviewer : 

1. What are your major products/services? (If you already know, then go to 
question 2) . 

2. How many people do you employ in this city or country? 

3. What new technology have you just brought or are you bringing into the 
company? (New technology - equipment, processes, management techniques, 
etc. ) 

4. What new technology are you planning to use in t ie next 3-5 years? 

5. What new skills or competencies do your employees reed? 

6. How are these skills/competencies developed? (ie. the skills listed in 
4). 
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Pago ?. 

ti/l Interview Survey 



7, What additional skills do graduates from our school need as they start 
work with your company'/ (If p<.;sible. refer to a specific graduate or 
graduates who have worked at this company). 



8, Ask how they work with local schools. Some examples are given below; 

a. Serve on advisory committees 

b. Teach a course (s) 
c» Visiting speaker 

d. Provide cooperative education work sites 

e. Provide work experiences for students 

f. Provide summer work experience for teachers 

g. Provide shadowing experiences for students 

h. Host student tours/field trips 

i. Other 

9. If they do not do any of the activities listed in seven, ask if they 
would be interested in one or more of these* 



10. Would your company be willing to serve as an intern (explain) site for a 
teacher this summer? (ie. Provide paid work experiences or opportunities 
to shadow/observe in the company). 
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APPENDIX B 
One and Five-Year Follow-Up Surveys 
* One-Year (BG-1) 
* Five-Year (BG-5) 
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CV.f Al HIGH . \ I CH 
(III "t\ u .\ UU I 



$ M ! v v. h a U . « Q llo vMi t»„ „s u ny i : y ... 



/ Vif7#* / 



V\;; .iff pie'.onlly doixuj an evaluation ol our school district's piufjram;; Wo want to improve these programs and also wish U) 

Your Inmost input will ho appreciated. 



determine job and odtjcalion.il preparedness ol nil graduato 
Student Naino 



I hgh Scnuol 



Main 



I (.'maid 



I am presently (I'loaao clwck c?/'l' of thoso loUowvuj 
sown statements } 

I nil time employed ;!? lus/weok or more; 

part It mo employed' loss Hum 3? hrs/weok. 

(1) onHoyed FULL lime in n nonmililnry occupa- 

lion. 

(2) employed F ULL lime in ihe military sorvico. 

(!?) employed PAUT lime--no( in school. 

(4) employed PAflT lime while in school. 

(5) full-lime slurient-n employed. (If chocked 

" go to tttS.) 

(6) unemployed but looking (or paid employment. 

(If checked go to #15.) 

(7) unemployed, nol looking (or paid employment. 

(If checked go to #15 ) 

Do you own or operate your own business? 



0)_ 

(3) „ 



Yes, lull-lime 
Yes, parMime 
No 



8. What is your present hourly wage including tips? (Do 
not include overtime) 

S /hour $ hours/week 

If you are not paid on an hourly basis, please respond 
as lollows: 



$ amount 

9. (I) Job Title 

(2) Job Duties 



number ol hours per pay ;>eriod 



(3) Zip code ol city in which you work 



10. May we contact your supervisor in our Employer 

Follow-up? (His/her response will be kept anonymous) 



(1) . 

(2) . 



Yes 
No 



(your signaturo) 



11. Supervisor's name: 
Business name: 
Address: 



12. Old your high school preparation help you to obtain your 
present job? (Check one) 

(1) Considerably 

(?) Somewhat 

P) Very little 

(4) None 

13. Is tins job relalod to your area of high school 
preparation? (Chock one) 

(M _ directly relalod 

(in same /oh .rrrvi uu high school preparation) 

(2) . ... somewhat related 

(umtuj some skills teamed in high school 
vow scs ) 
... PUl rolaiod 



(f or School Use) 
[lPO}:>l OU; NCI ] (jf] 



\\v moiiii 



Ml: 


Tf 







AIM 
\ 



IK) 

;t 

Ai'L 



IVNM 



14. Whal f #;uls of your high school preparation have been o! 
greatest value in your present job? (Check tluit apply) 



(0 

(2). 

(3) 

('«). 

(5) 

(6) 

(>) 

(8) 

(9) 



General job skill*; (work habits, working with 

others, etc ) 

Spor.ilic job skills 

Career awareness and information 

General preparation 

English skills 

Maih skills 

Science skills 

Computer skills 

Other 



1 13. Mow do you feel about the courses you took al your high 
school? (Check one) 



(2) 

(3)_ 
(<)_ 



I was very Salisfied 
I was Satisfied 
I was UNsatistied 
I was very UNsatisfied 



16. Do you (eel vo .nal courses including career explora- 
tion and the development of saleable job skills should 
be part ol the high school program? 

(1) Yes 

(2) Not sure 

(3) No 

17. Are you current!/ attending a school/educational 
program? (Check one) 

V 1 ) a C£D (High School Equivalency) program 

(?) a vocational-technical school diploma 

program ona-ycar or less 
(3) a vocational-technical school two-year 

associato degree program 
(*) a two-year UW-Center program 

( 5 ) a four-year university or college program 

(6) an apprenticeship program 

( 7 ) an in-service or on-the-job training program 

W private school program 

(9) I am noi attending school 



(10) „. 



other 



10. If you checked a school/educational program in #17, 
please list your current program/area of study. 

Name of currcWprogw 

Name of school location 



I'ln, is« turn ihhjo 
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In general, how adequate was your high school propnialion in Iho following siluadons or lilo skills. (t'loastt cmihi number !*, K J. 
.?, Q( I for each statement ) 



Mote Than 
Acd.Kiii.-iii! AdLH Jl': ,u i_ 



Noi 
Adequate 



1') 

::u. 
i 1 1 

;) ( > 

;m 

'. *' > 

;v. 

20 

30 
31 



Seeking a jot i J 

Learning sponlu: job -..Kill.. J» 

l^Mrning speolu: job knowledge !» 

(ioUtne along wilh others •* 

IUniu.i able in talk to Iho Ihi:;s about your pmhlem.-. !» 

UiHleisMndmg the valuu and importance o! work !> 

Solving problems - 1 

Understanding flu* II !i and v/oikl economic systems (supply -demand. 

inllalion. nxression. olo ) 

Understanding how a business makes a profit t> 

Understanding alternatives lor solving homo and family rolalod problems S 

Understanding child growth and development and parenting responsibilities. . 5 

Managing career responsibilities wish home and family responsibilities 5 

Applying basic skill preparation in math, reading, writing, spoiling, speaking, 

listoning to every day problems **> 



:» 

3 
3 

3 
3 
3 

3 
3 



In general, how adequate was the career planning assistance and information you received at your high school? (Please citclo 
number 5. 4. 3. ?.. or I lor each statement). 



Mora Than 
Adequate 



Adequate 



Not 
Adequate 



32. Determining my skills, interests and abilities 5 

33. Opportunity to hear speakers from the world ol work 5 

34. Presentation of role models in nontraditional areas (i.o., femalo 
mechanics/male nurses) 5 

35. New and/or future oriented occupations o 

3G. Jobs held by primarily males or primarily females 5 

37. Occupations which have local job opportunities 5 

38. Jobs which become outdated, overcrowded or oiler no opportunities for 
advancement 5 

39. Post-high school educational opportunities related to my career goal 5 

40. Job alternatives related to my career goal 5 

4 1 . t tow to find a job/how to apply tor a job 5 

42. How to start a business 5 

43. Selecting a career i> 

44. Adapting to change 5 



4 
4 

4 
4 
4 
4 

4 
4 
4 
4 
4 
4 
4 



3 
3 

3 
3 
3 
3 

3 
3 
3 
3 
3 
3 
3 



2 
2 

2 
2 
2 
2 

2 

2 
2 
2 
2 
2 
2 



t 



45. What is your home zip code? . 



46. What additional skills and knowledge could you have used as you won/ from high school to work or further education? 



4 7. Yes No Would you tiko a copy ol the 

survey summary? II yes. please give your name and 
mailing address. 



Nanu 



City 



State Zip _ 



lltanh you lot responding. Please return this completed survey in the enclosed prepared envelope, Add any additional 
comments on the back ol the covet letter and also return 
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J: l>JJ:-;_Y..t:A l A .</< JJ .(<o\v.iir m i u v i: v 



Wi! art! presently doing an evaluation ni our school district's programs. W»i want to improve these programs and also wish to do- 
lorminn job and educational preparedness of all yraduaios. Your honor,! input will ho nppioaaled. 



student Name 



I ittjh School 



UTAIU Ml. Ml-, 
•j (1) KUWi 



('.') female 



(> I am presently (Ptoaso chock one, of these following 
now: i staten u }nts ) 

full-lime employed: 32 hrs/woekor nioro; 
part-lime employed: loss than 32 hrs/weck. 

(!) _ ... . employed rui.L ume in a nonmihtary 
occupation. 

('?.) employed FULL timo in the military service. 

(3) employed PART lime-not in school. 

(4) employed PART time while in school. 

lull lime studont-nol employed, {li chocked 

go to #15.) 

(6) unemployed but looking lor paid employ- 
ment (tf checked go to #15.) 

(7) unemployed, not looking lor paid employ- 
ment, (if checked go to #15.) 

7. Do you own or operate your own business? 

(1) Yes. toll-time 

(2) Yes, part-time/second income 

(3) Z No 

8 What is your present hourly wage including lips? (Do not 
include overtime) 

S /hour It hours/week 

II you are nc 1 paid on an hourly basis, please respond as 
follows: 



S amount 

9. (1) Job Tale _ 
(2) Job Duties 



number ol hours per pay period 



(3) Zip code ol the city in which you work 



10. May we contact your supervisor in our Employer Follow* 
up? (His/her response will be kept anonymous) 



(1) . 

(2) . 



Yes 
No 



(your signature) 



1 1. 



Supervisor's name: 
Business name: 
Address: 



12. Is this |Ob related to your area of high school 
preparation? (Check one) 

(1 ) directly related 

(tn same job area as lugh school preparation 

training) 
(?) somewhat related 

(using some skills teamed in high school 

courses) 
(3) no] related 



(f or School Use) 
I [^h^^^f^l ™ ] 

hi: 



M 
t 

AlH 

i 



I to 

3 

no 
:t 



Ml' 


w. 







Air 


liNSI 


111 3. 




3 


4 



VVNi i] 



14. What parts of your high school preparation have been of 
greatest value m your present job? (Check all that apply) 



Cionoral job skills (work habits, working with 
others, eic) 
Specific job skills 

Caieer awareness and information 
General preparation 

liltfjIiSh Skills 

Math skills 
Science skills 



m _ 
(?) 

m 
pi 

(G) _ 
(?) _ 

(B) Computer skills 

(9) Other 



15. How do you leel about the courses you took at your high 
school? (Check one) 

(1) I was very Sa'islied 

(2) 1 was Satisfied 

(3) I was UNsatisfiod 

(4) I was very UNsalislied 

1G. Do you leel vocational courses including career explora- 
tion aid the development ol saleable job skills should 
be part of the high school program? 



0). 
(2). 
(3) 



Yes 

Not sure 
No 



17. Since leaving high school, (Check all that apply) 

(1) I have received one or more job promotions 

(2) I have changed jobs one or more times 

(3) I am doing the same woik that I did 

immediately following high school graduation 

18. Since leaving high school, I have completed or am 
presently enrolled in: (Check all that apply) 

(1) a GHD (High School tquiva^ncy) program 

(2) a vocational-technical school diploma program 

one year or less 

(3) a vocational technical school two-year 

associate degree program 

(4) a two year UW-Conter program 

(5) a four-year univo 'y system or college program 

(G) an apprenticeship program 

(7) aii in service or on-the-job training program 

(8) private school program 

(9) I have not taken any additional training 

(10) other _ 

19. II you checked a school/educational program in 018, 
please list your most current program/area ol study. 
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13 Did your high school preparation help you to obtain your 
present job'' (Check one) 



(i) 

( i) 



<;onMdi.'Mlily 
Sonipwlui 
Vt'iy Mile 
Mom' 
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Name of current pfogram/atea of study 
Njmo ot school loCiitton 
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(/// A'.'M M / Af^i.i 



In general, how ;uliM|u,ih; w.is yom high school preparation tn (ho following situations or lilo skills: (l*leaso arch number 5, 4, .1, 
l\ 0( 1 lor onclt statement ) 

Mom 1 1 k in Not 
Adoqualo Acfi.K|i j.ifc? Adequate 



? 1 t (Milling S|)'*i:ilii: ji)h r.kilK i, ,j j ^ | 

I earning '.pectin: job knowledge », 4 -j ^ ^ 

Gelling along wilh other; !, ,\ ^ j 

;M. Heine) able to talk lo Iho boss abuul your piobloms !, 4 3 ;j j 

?\). Understanding the value and im(M>ilani:o ol woik !, a [\ ? \ 

iHi Solving piotiloms i, ^ j ^ j 

H/. Undorslnndinq the U S and world economic systems (supply-demand. 

inflation. r<\:ession, etc ) c } ^ ^ \ 

Understanding how a business makes a prolil 5 4 ;j 2 \ 

29. Understanding alternatives lor solving homo and faintly related problems 5 a :\ ? \ 

30. Understanding child giowih and development and parenting responsibilities. , 5 A 3 2 1 

31. Managing career responsibilities with homo and family responsibilities 5 A 3 2 1 

32. Applying basic: skill preparation in math, reading, writing, spotting, speaking, 

listening to cvfiiy day problems * 5 4 3 2 1 



In general, how adequate was the career planning assistance and information you received at your high school? (Ptoaso circle 

number 5. 4. 3. 2, or / tor each statement). . 

More Ttian Not 

Adequate Adoqualo Adequate 

33. Determining my skills, interests and abilities 5 4 3 g j 

34. Opportunity to hear speakers Irom the world ol work 5 a 3 2 1 

35. Presentation of role models in nonlraditional areas (i.e., female 

mechanics/male nurses) 5 4 3 2 1 

36. New and/or future oriented occupations 5 4 3 2 1 

37. Jobs hold by primarily males or primarily females 5 4 3 2 1 

30. Occupations which have local job opportunities 5 a 3 2 1 

39. Jobs which become outdated, overcrowded or oflor no opportunities lor 

advancement ^ 4 3 2 

40. Post-high school educational opportunities related to my career goal 5 4 3 ? j 

41. Job alternatives rolatod to my career goal S 4 3 2 1 

42. How to find a job/how to apply lo. a job 5 4 3 2 1 

43. How to start a business 5 4 3 2 1 

44. Selecting a career 5 A ^ 2 j 

45. Adapting lo change 5 4 3 ^ \ 

4G, What is your homo zip code? 



*7. What additional skills and knowledge did you have to learn in order to keep up with changes in your job. 



W. Additional skills or knowledge you will need lo learn in the near future in order to keep up with changes in your job. 



49 Yes No Would you like a copy ol the 

survey summary? II yes, please give your name and 
mait'ng address. 



Name 



Cily 



Stain 



Uuwk you for responding />/,,;isr? return this completed survey m the enclosed prepared envelope Add any additional 
comments on the back ol the cover letter and also return 
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APPENDIX C 

Employer One and Five-Year Fol low-Up Survey Forms 

* Employer One-Year Fol low-Up Study (Form CI) 
*Employer Five-Year Follow-Up Study (Form C2) 
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( VTAI7DPI 

/// we/ / 



Deaf Super vts<n f Employe/ 



We ate in the pn tress o) cvtdwitiny out school dtstttct's vocational ptoyjams and have icceived written waiver pom \ottt employee to contact 
Mm Since this wtllpntvidc valuable data lot out diMnet with respect to pto^tam iitipt,nctnent tis well as fob and educational preparedness oj 
graduates your honest input is vital ) om cooperation will be appreciated No student, employe/ at identifiers will he presented in out 
reports Stan with item / 



Lmployou 
High School 



Area o1 High School Vocalional Preparation 



(hoe School Use) 
iPC {sUQUl- NaT] ] 

ACi|mnii)jnii}Nii l inj 
i l 2 iTlTI s I (71 



4. 



6. 



Malcl Pcmalc 



Ci ra duatc 



5. [WFiiH 

G raduatc | 





API 


BNH 


inslwNii] 




2 


3 





Please rate the individual in the following areas: (Circle the appropriate responses) 
START HKKK 



Well Adequately Not at all 
Prepared Prepared Prepared 
(5) (4) 



7. Occupational knowledge 

8 Occupational skills 

9. Work attitude 

10 Woik quality ... 

I 1 Work quantity (amount produced) 



5 
5 
5 
5 
5 



4 
4 
4 
4 
4 



(3) 


(2) 


(1) 




2 


1 


3 


2 


1 


3 


2 


1 


3 


2 


1 


3 


2 


i 



In the following areas of basic skills, how well prepared was this employee? (Circle (he appropriate 

responses) 



Well Adequately Not at all Not 
Prepared Prepared Prepared Observed 

(5) (4) (3) (2) (1) (0) 



5 4 3 2 1 0 
5 A 3 2 1 0 



1 2 Reading 

VS. Spi'lhti)' 

1 1 Computer literacy 

15. Math 

16. Science 5 4 3 2 1 0 

17. Writing 

IS. Speaking 

19 UstenitM; 

20 Problem solving 

CD t- 41 



5 4 3 2 1 0 
3 1 3 2 I 0 



S 4 3 2 I 0 

4 3 2) () 

5 1 y 2 I 0 

S 4 3 2 I 0 



in /?v/cry 



K«^!M,(>VKK.: /INK-YKAU FOU.OW-I'I' STUDY 

Siroiulaiy Vocational iN()^;im1ivnlii:i!ion " 



l 7 avcJ 



IMense rate this employee on each of (ho characteristics listed below. (Circle (he appropriate 
responses) 



Well Adequately Not at all N ol 
I'rcp.ucd Prcpaicd Prcpaicd Ohscivcd 

(5) M) O) (2) (I) (0) 



21. Use of tools and equipment 

22. Selection and care of space, materials, and supplies. 

23. Ability to work with co-workers 

24. Accepting advice and supervision 

25 Working in a leadership role when applicable 

26. Understanding the value and importance of work. . . 

27. Prompt and dependable in attendance 

28. Willing to accept and pcuorm tasks 

29. Ability to plan and direct own work 

30. Ability to make sound decisions 

3 1 Shows consideration and kindness towards others. . 

32. Neat, rican, and appropriate appearance 

33. Flexibility to accept change 

34. Ability to learn new job tasks 



35. What is your overall rating of this individual's occupational 
competence as it relates to his/her job? 

(5) Very Good 

(4) Good 

(3) Acceptable 

(2) Poor 

(1) Very Poor 

36. How would you rate this employee's preparation in relation to 
other employees in his or her work grou^? 

(3) . Individual is better prepared 

(2) Both arc about the same 

(U Individual is less prc[ . red 

(0) No basis for comparison 

37 . What has been this employee's attitude toward on-the-job 
training? 

(5) Very positive, eager to learn 

(4) Positive, willing to learn 

(3) Acceptant, but needs prodding 

(2) Negative, resists training 

U) No training has been offered 

38. If the opportunity should arise, would you promote this 
employee? 

(D_ - Yes 

(2) Not sure 

(3) No 



39. When hiring recent high school graduates, wou 



0 
0 

0 
0 
0 

(1 

0 
0 
0 
0 
0 
0 
0 
0 



d you prefer to 



hire persons who have had high school vocational training? 

(3) A good deal of preference 

(2) Some preference 



(I) No preference 

40. Do you feel vocational courses including career exploration and 
the development of saleable job skills should be pan of die high 
school program? ^ 



(1) . 

(2) . 



.Yes 

. Nol sure 



(3) No 

41. In which of the following ways docs your business work with 
secondary schools to support education? (Please check all tluit 
apply) 



0). 
(2). 



.Suggesting new vocational courses 
. Recommending equipment and materials to be 
used 

(3) Providing equipmcnt/materials/donations 

(4) Providing facilities 

(5) Providing vocational students with work experi- 
ence 

(6) Serving on advisory committees 

(?) Providing teachers with work experience 

(S) Releasing employees to teach vocational courses 

(9) Partnerships 



Please comment on; 

(1 ) Additional skills you Feel are needed by graduates 



(2) How high school vc rational programs could be improved 



(3) What new technologies arc coming into your work place 
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Sccoiuliiiy Vocational Piogram Evaluation 



J'OK.M CI 

rave I 



Dear Supet visor I Employer, 



We ore ui the process of evaluating our school district's vocational ptoyjoms ami have tecennl written waiver from your employee to contact you. 
Since this will provide valuable (lata jor our district with respect to prostata improvement as well as job and educational preparedness of 
graduates your honest input is vital. Your cooperation will be appreciated. No student, employer or identifiers will he presented ui our /■<•/, 



Start with item 7. 



reports. 



I: mployco 
High School 



Area ot High School Vocational Preparation 



(Fa* School Use) 



IPC 



S KQlll-Nei- 1 Gl7] 



2. 


AG 


m 


ID 


im 


Ml- 


11: 




1 


2 


3 


4 


5 


6 


3. 


11 


DL 


ID 









4. 



Male! Fema le 
Graduate 



5. 



pv I rr 



Graduate 



AIN 


API 


BNH 


HIS 


WNH 


1 


2 


3 


4 


5 





Please rate the individual in the following areas: (Circle (he appropriate responses) 
START HKRK 



Well Adequately Not at all 
Prepared Prepared Prepared 
(5) (4) (3) (2) (1) 



7. Occupational knowledge 

8. Occupational skills 

9. Work auitudc 

10. Work quality 

11. Work quantity (amount produced) 

In the following areas of basi 
responses) 



12. Reading 

13. Spelling 

14. Computer literacy 

15. Math 

16. Science 

.17. Writing 

18. Speaking 

19. Listening 

20. Problem solving . 



5 


4 


3 


2 


1 




5 


4 


3 


2 


1 




.S 


4 




2 


1 




5 


4 


3 


2 


1 




5 


4 


3 


2 


1 




this employee? (Circle the appropriate 


Well 
Prepared 
(5) 


(4) 


Adequately 
Prepared 
(3) 


(2) 


Not at all 
Prepared 
(1) 


Not 
Obsen 
(0) 


5 


4 


3 


2 


1 


0 




4 


3 


2 


1 


0 


5 


4 


3 


2 


1 


0 




4 


.■> 


2 


1 


0 




4 


3 


T 


1 


0 


s 


•1 


3 


') 


1 


0 


s 


■1 


3 


2 


1 


0 


s 


A 


3 


> 


1 


0 


s 


A 


3 


> 


1 


(1 
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kM'LOYK^ 

Secondary Vocntional Yrop.ram HvaTiKuinn 



J'orm £1 



lMcasc rate (his employee on eacli of tiie characteristics listed below. (Circle (he appropriate 
responses) 



21. Use of tools and equipment 

22. Selection and care of space, materials, and supplies. 

23. Ability to work with co-workers 

24. Accepting advice and supervision 

25. Working in a leadership role when applicable 

26. Understanding the value and importance of work. , . 

27. Prompt and dependable in attendance 

28. Willing to accept and perform tasks 

29. Ability to plan and direct own work 

30. Ability to make sound decisions 

31. Shows consideration and kindness towards others. . 

32. Neat, clean, and appropriate appearance 

33. Flexibility to accept change 

34. Ability to learn new job tasks 



veil 
:paiv 
(5) 


Adi 
1 IV 
('1) 


•(|iiaicly 
epared 
0) 


(2) 


Not at all 
Prepared 
(I) 


Not 
Ohscivc 
(0) 


S 


A 


:\ 


2 


1 


0 


S 


A 


3 




I 


0 


5 


A 


3 


2 


1 


0 


b 


A 


3 


2 


I 


0 


5 


A 


3 


2 


I 


0 


5 


A 


3 


2 


I 


0 


5 


A 


3 


2 


I 


0 


5 


A 


3 


2 


I 


0 


5 


A 


3 


2 


1 


0 


s 


4 


j 


2 


1 


o 


5 


4 


3 


2 


i 


0 


5 


A 


3 


2 


i 


0 


5 


4 


3 


2 


i 


0 


5 


4 


3 


2 


i 


0 



35. What is your overall rating of this individual's occupational 
competence as it relates to his/her job? 

(5) Very Good 

(4) Good 

(3) Acceptable 

(2) Poor 

(1) Very Poor 

36. If the opportunity should arise, would you promote this 
employee? 



(1) . 

(2) . 

(3) . 



.Yes 
Not sute 
.No 



38. In which of the following ways docs your business work with 
secondary schools to support education? (Please clicck all that 
apply) 

(1) Suggesting new vocational courses 

(2) Recommending equipment and materials to be 

used 

(3) Providing cquipment/matcrials/donations 

(4) Providing facilities 

(5) Providing vocational students widt work experi- 
ence 

(6) Serving on advisory committees 

(7) Providing teachers with work cxjx:ricncc 

(8) Releasing employees to teach vocational courses 

(9) Partnerships 



37. When hiring recent high school graduates, would you prefer to 
hire persons who have had high school vocational training? 



(3)_ 
(?•)_ 

(1) No preference 



A good deal of preference 
Some preference 



Please comment on: 

(I) Additional skills you feci are needed by graduates 



(2) How high school vocational programs could be improved 



(3) What new technologies are coming into your work place 
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APPENDIX D 
MODULE EVALUATION FORM 
* Project Staff Form 

* Teacher Form 

* Student Form 
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i: SAMl. 

Cil lt»: i M«:«mI*.: Improvi-mcni 
< Sa i i •.. I <i«: t ory 
.t • Very Oood 



D imens i on 



L. TITLE: 

The modulo contains a name 



!i | | ! i H I U.\ IMM' !'!<" »« 1 

I'RO.i K1VI" S'l' A F K 
:ionii!.i; !-:v,\i.iiA'i'ii)N i-oi:,m 



Rati oh 



Comments 



3. 



DESCRIPTION : 

Information is provided as to where the 
module fits in a sequence of a unic/task/ 
course * 

OBJECTIVES: 

A. Module objective is stated 

B. Specific objectives are listed. . . - 

CONTENT OUTLINE: 

first, second and third order outline 
,f the content is presented 



6. 



ACTIVITIES: 

Are identified which will help students 
learn the content 



RESOURCES : 

A variety is listed which will help 
students/teachers accomplish objective 



1 2 3 



1 2 3 
1 2 3 



1 2 3 



12 3 



12 3 



7. EVALUATION : 

A. Evaluation criteria identified 

B. Evaluation procedures given . 



12 3 
12 3 



ADDITIONAL COMMENTS: 
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IT.ACIlKk KVALUA11 wN 
II li.ll YiWM muuuu; 



V,/ /.W MM 



UAH-; 



,a.K rrn.K 



I)ATK(s) USED 



CLA:>S(<:«; ) 



TIME ic llijflJ t Kill J Tu UJMIM.KTK THE MuUULK (MINUTES) 

Direct .ion:; : Please list, the ittl ormat ion reqtio!; i ed above, Tlu:n read each «; i a i:««cut 

below and respond based ou your experiences with lite module. Use the 
t o 1 1 ok i [\\\ response!; . 



I - SI) - Strongly Disagree 
?. = 0 - Disagree 

3 - U ~ Undecided 

4 - A ? = A^ree 

5 = SA = Strongly Afcree 



If you have any comments, write them by the statements or at the end of the for 



EVALUATION STATEMENTS Sfj 

„ l__ 

MODULE 

The Module was easy to use... 1 

The Module was complete 1 

3. Objectives were clear 1 

4. Activities in the Module were effective in 
stimulating learning 1 

5. Audio visual aids and/or resource materials 
were effective I 

6. I had the classroom and lab equipment 
required to effectively use this Module 1 

7. Evaluation procedures were .effective 1 

6. This Module introduced new content. 

concepts, skills, etc. in wy class 1 

9. The knowledge and/or skills students 
'learned are relevant to the needs and 
trends in industry 1 

10. This Module fit well with the rest 

of my course content 



Re sponses /Comments 



D 

2 



2 
2 
2 



2 
2 



U 

3 



3 
3 
3 



3 
3 



A 
4 



4 
4 
4 



4 
4 



SA 

5 
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fl i t ill Toch (10/10) 



STUDENT EVALUATION FOKM 



school 



OAT I - ! 



:lass 



module 



Direct io»r. : 



Road the statements on the leCt. Use your; experiences in the 
unit: just completed in class to answer each statement. Use the 
following responses. There Ls space Cor comments at the end. 

1 «= SD = Strongly Disagree 

2 - D ~ Disagree 
"- 3 ~ U - Undec ided 

4 - A = Agree 

5 = SA - Strongly Agree 



EVALUATION STATEMENTS SD 

1 t liked this learning unit.l 1 

: I learned some useful things .1 

3. The learning activities were interesting 1 

4. The activities, tasks, and/or assignments 

were helpful * - 

5. I learned some things that will be useful 

in the future * 

6- I learned some new things 1 



Responses 
D U A SA 
2 3 4 5 



2 
2 
2 



2 
2 



3 
3 



4 
4 



4 
4 



5 
5 



5 
5 



7. What was the most important thing you learned? 



8. EIow could this unit be improved? 



0 
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ENVISION 
Introduction 

It takes a concerted effort for schools to keep current with 
the technology and systems used in business and industry. 
Vocational and technical programs have a special challenge to 
develop the requisite competencies, skills and knowledge, needed 
to adopt to and utilize the technology that businesses and 
industries have purchased or plan to purchase in the predictable 
future. Educators are also vitally concerned with the future. 
In many instances it may be two to four years before their 
students have an opportunity to use the competencies and skills 
in the workplace, when a vocational or technical program is 
similarly based on competencies that are being phased out in the 
world of work, students have a difficult time finding employment 
and, when they do, their initial productivity is less than 
expected, based on the level of training. 

Educators need techniques and processes that will allow them 
to look ahead, to envision what will be in the workplace two, 
five, and ten years in advance. The ten year picture may be 
somewhat fuzzy, however, educators need to be able to look ahead 
two and five years with some clarity. The people involved with 
training in business and industry also need techniques that will 
allow them to look ahead at the types of skills and competencies 
required to keep their companies competitive in a world 
marketplace. The traditional task analysis techniques provide an 
appropriate way to determine the skill and knowledge content of 
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jobs that are relatively stable. However, when the skill content 
of a job is changing, the task analysis process always lags the 
actual training needs. In other words, it cannot predict what 
competencies and knowledge are needed to perform on a job that 
does not exist at this point in time. 

This fact became evident when the High-Technology Training 
Model Project staff members worked with a large company that was 
developing a new production model and a new production system. 
Through adaptation and experimentation with a combination of job 
analysis and futurist techniques, project staff members were able 
to develop a process that worked successfully. No doubt, this 
process will need to be refined as it is applied in other 
■ettings . 

This module will share the envision process with the reader. 
Readers are encouraged to apply and modify it as necessary to fit 
their particular applications. 

Purpose 

The purpose of this module is to present several techniques 
and processes that can be used to envision the competencies and 
knowledge needed in jobs that are evolving and changing. 

Ob-i ectivss 

This project will meet the following objectives: 

1. Identify situations in which the envision process should be. 
used. 

2. Describe how the selection of a proven system can provide 
the framework for identifying training needs. 

52 

r>2 



3. Describe techniques that can be used to create a vision that 
can be used to transform an organisation. 

4. Identify ways to overcome reluctance to change. 

Background 

This mocVule is ba&ad on the experiences of staff members on 
the High- Technology Training Model Project, it grew out of the 
realisation that traditional needs assessment techniques do not 
wor:K effectively wtu . the production systems, jobs, and/or 
management systems r/ y~ m a company are in transition. Often 
when compa-'ifts re^lizw *M«y need to change, they feel there is a 
need to -cr * in their araployees. However, it is difficult to 
identity the type ~£ restraining required. This situation also 
applies to educational institutions. 

In two companies the project assisted management staff 
members* who wanted to adapt and employ proven production systems. 
Given the nature of their product lines, this approach appeared 
to be appropriate. The major challenges were to select the 
appropriate production system and make a commitment to 
implementing it. in a third company, a new product line and a 
new automated production system were being developed. In this 
setting, an existing system was not available to guide the work 
of the training developer. 

Project staff worked with trainers and supervisors in ail 
three companies, in the first two companies, the adoption or 
deployment of an existing system provided an appropriate 
framework for designing training and identifying the types of 
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competencies and knowledge needed. In the third company, more 
unique techniques and processes had to be devised. 

Another facet of training in business and industry that 
needs to be considered is relevancy. Most managers, supervisors 
and line employees want to see the relevancy of the training they 
are taking. Based on our experiences in these companies, it was 
apparent that unless the training was relevant, employees often 
lost interest and in some cases quit attending. When training 
programs are developed to prepare employees for future tasks and 
roles, it is important to communicate how these will fit into the 
new types of jobs and tasks envisioned in the company. This 
process will also be important in school programs that enroll 
students who wish to have relevant instruction and want to know 
how the contents of their learning activities relate to what they 
will be doing in the world of work. 

The techniques and processes that follow are based on a 
variety of experiences in assisting schools and companies in 
identifying educational and training needs. Each technique has 
strong and weak points. You will need to review them and select 
the most appropriate one for your needs. 

Training Process Continuum 

The stages in the training process continuum are depicted in 
Figure 1. Each of the stages has a purpose and value. The 
starting point, perspective and outcomes for each stage vary. 

The four stages are show in a hierarchical sequence. This 
was done to communicate that these stages do have an inter- 
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relationship. For example, the knowledge acquired through short- 
term, very focused training, in the find and fix stage can be 
used to design longer range training programs for the performance 
improvement stage . These connections will be further illustrated 
as each stage is described. 
Find and Fix 

Training and education in the find and fix stage is done to 
solve immediate problems. Often this is described as putting out 
fires. The training is focused on specific problems and is 
designed to provide the best program under the given time and 
resource constraints. In some respects this can be viewed as 
Just in Time (JIT) training, it does help to solve immediate 
problems and improve performance. Also, it can provide knowledge 
for a systematic training program. However, primary emphasis on 
this approach delays the development of an organized and 
systematic training program. The information gained in this 
phase needs to be used in designing a more comprehensive training 
program . 
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Improve Performance 

The improved performance stage emphasizes a systematic 
training program focused on developing all employees' skills and 
competencies. The goal is to improve the efficiency and 
Figure 1: Training Process Continuum 
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effectiveness of the organization. The training program is 
designed within the current technology and systems used by the 
company. A task analysis process is used to identify the skill 
and knowledge levels needed by various groups of employees, m 
addition, feedback from a quality program can help to identify 
product, service and process defects that can be remediated 
through training and education. 
Make A Transition 

Frequently a company will find that it's existing technology 
and systems are not adequate to make it competitive. There is a 
need to make a transition to new technologies and systems, in 
the make a transition stage, the emphasis is on selecting proven 
technology and systems to implement in the company, one of the 
best examples of this during recent years has been the adoption 
of quality programs in a large number of companies in the United 
States. Although a major change is being made within the company, 
a proven framework for this change exists in other companies, m 
other words, the technology and systems have been proven to be 
functional in a variety of other settings. The major challenge 
to the company involved in a transition is to select the 
appropriate technology and system ( s) , prepare its employees to 
use them, and provide appropriate support for the infusion of the 
new technology/system, since the technology and systems are 
known, a task analysis approach can be used to identify the types 
of skills and knowledge needed to manage and operate the system. 
This information can then be used to design the training program. 
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In addition to training on these skills and knowledges, attention 
will need to be given to the change process and the types of 
orientation/training activities required to facilitate change. 
Create A Transformation 

The fourth level, create a transformation, is utilized when 
an organization must make significant changes to remain 
competitive. At this stage, an existing system is not available. 
The company must study its needs, monitor new developments and 
construct a valid system. The following sections describe in 
more detail techniques that can be used in the envisioning 
process. 

Envisioning Process 

The envisioning process requires the input of information 
from a variety of sources, processes for creatively analyzing 
this inf< ' r.tion, and consideration of the organization's 
culture. Each of these will be described in more detail in the 
sections that follow. 
Information for Envisioning 

Two types of information are used in the envisioning 
process. First, one needs to be aware of technological advances 
and the development of new management systems. Second, a 
knowledge of the specific company, plant site, or organization 
within which the transformation will take place is required. A 
scanning process is one technique that can be used to acquire the 
first type of information. Processes for collecting 
organizational information are described in a later section. 
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Scanning Techniq ue 

The scanning techniques should allow one to tap a variety of 
resources and sources of information. As many staff members as 
possible should be involved in the scanning. For instance, each 
might be responsible for monitoring the technological trends 
reported in the literature they read and at the conferences they 
attend. This would provide a multiplier effect and allow the 
organization to have a broader perspective than what would ba 
attainable through having just one person responsible for 
scanning. Periodically, the results of the scanning activities 
should be discussed in staff meetings or other small group 
sessions and synthesized to identify potential changes, if this 
is systematically done it would allow one to identify trends and 
pending new developments. Several techniques that can be 
employed to scan new developments are described below. 

Interacti ng with Business and Industry - 

One way for educators to monitor changes in technology and 
new systems implemented in business and industry is to maintain a 
close and continuous contact with people in area companies. One 
department in a company that has first hand knowledge of training 
needs and the new competencies required is the training 
department. Some companies are expanding this function and 
calling it human resource development (HRD) . in either case, the 
people involved in these departments/units are concerned about 
identifying the competencies needed to operate the technology 
used in the company and effective ways to develop these 
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competencies. Many small and medium sized companies have not 
appointed someone to the responsibility of training director or 
human resource development director. In a small company this 
responsibility is sometimes retained by the president or is given 
to a manager. In other instances, small companies are not 
systematically involved in identifying their skill and competency 
requirements or in training. 

Area meetings of professional organizations provide an 
excellent opportunity to network with training professionals, 
managers and technical staff. Presenters at these meetings 
usually discuss innovative technology and systems. National 
conferences provide a expanded opportunity to interact and learn 
more about new developments. Another way for educators to 
interact with B/I is to visit and interview employers in area 
companies. This process and sample interview questions are given 
in the module on B/I interviews. Advisory Committees can be used 
as another effective way to acquire input form B/I. 

Published Materials . 

Newspapers probably have the shortest lag time between the 
advent of a new technology/ system and the time when information 
appears in print on this development. Usually newspapers do not 
give an indepth treatment of new developments and systems. Also, 
some papers have better science, technology and trends sections 
than others. Newspapers are, however, useful in providing early 
signals on the development and deployment of new technology and 
systems . 
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Several trend letters are published for various audiences. 
These are often useful in monitoring new developments. 

It takes longer for an article to be developed and published 
in a journal than in a newspaper. However, journal articles 
usually provide more depth and insight into the technology and 
its applications. Staf£ members should be responsible for 
perusing or reading professional journals related to their 
disciplines. In addition, a review of research journals related 
to your organization's work i lid be done. The trends/new 
developments columns in journals usually contain information that 
is worthwhile. These columns should be reviewed systematically. 

Information in books is usually less current than that 
available in journals. Books have an advantage in presenting a 
more indepth presentation on a particular technology or system. 
More information is provided on the technology/systems and its 
applications. 

All of these published materials can be helpful in 
monitoring technological, systems and societal trends. It is 
very difficult for any one person to be able to keep up with all 
of the information that is published. Therefore, each member of 
an organization should be given responsibility for monitoring 
certain types of journals, selected newspapers and specific 
books. Every four to six weeks, staff members should list and 
discuss the trends and new developments they have found. 

Labor Market Information . 

Certain types of labor market information can be* useful in 
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identifying new technological developments and systems. The 
adoption of new technology and systems is often accompanied by 
the generation of new job titles and/or significant changes in 
existing jobs. Also, the impacts of new technology may be 
reflected in a declining number of jobs in some areas. For 
example, the widespread us of personal computer^ (PC's) and word 
processing software has led to the development of word processing 
jobs and a decrease in the number of typists hired. Federal and 
state governments produce several documents on a monthly and 
annual basis that provide summary data on the labor market. 
These should be scanned on a regular basis to isolate any trends 
that are occurring. 

Another source of information on changing technology and 
operating systems can be found in the position descriptions 
posted in professional journals and newspapers. In addition, the 
types of jobs listed are an indicator of changing technology and 
the competencies required in the workplace. In the 1950' s and 
60 's keypunch operators and computer programmers were growing in 
number, now key entry jobs have replaced keypunch operators and 
computer systems people are preferred to programmers. Often the 
spread of new technology can be observed in the number of new job 
titles created and the increasing numbers of fields in which the 
technology is used. Two contemporary examples of this are 
computer and laser technologies. 

Special studies are frequently conducted by national 
associations, federal and state governments, and local economic 

62 



72 



development associations. For example, Profile 21 commissioned 
by the Society of Manufacturing Engineers identified new roles 
for the manufacturing engineer and new competencies for this 
position. These studies are usually stimulated by a changing 
labor market and a changing skill composition in jobs. Thus, 
they can be useful in identifying new competencies. 
Data From Your Organization 

Quality, reliability and production data can be usefu". in 
identifying problem areas. The data can be displayed and 
analyzed using a Pareto Diagram? to identify the most common and 
costly problems. These problems can then be further analyzed 
through the use of cause and effect diagrams to identify 
potential causes related to the prcblem. 

An organization's customers also provide useful information 
for a company. The customer would be the individual or 
organization that purchases the organization's services and 
products, one form of feedback is customer complaints. However, 
many people do not complain. Instead they switch to a 
competitor. Therefore, it is important to take a pro-active 
approach and contact your customers. Interviews and surveys can 
be effectively used to identify what is being done right and what 
needs to be changed. These contacts can also be used to obtain 
information on new products and services the customers would 
like. 

In educational situations, students, graduates and employers 
are the customers. These people need to be contacted to obtain 
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feedback concerning the quality and relevancy of the educational 
program (s) under study. Other modules ir this handbook describe 
graduate and employer follow-up studies. 

One of the problems often encountered in U.S. companies is 
the separation between the research and development (R&D) , and 
manufacturing/production functions. This situation is often 
described as "The R&D Department throwing a new design over the 
fence to the production department." This approach does not lead 
to a smooth transition from the design stage to production. 
However, the solution of this problem is not the prime focus of 
this module. Instead, the intent is to make the point that 
information and concepts developed during the R&D, and prototype 
stages can be useful in developing the vision for the new 
production system and the type of training required. Trainers 
need to be actively involved in these early stages. 

This conclusion is based on the experiences project staff 
had with a company that was designing a new product. In 
addition, the company involved was working on a new production 
system that was much more automated than the current system. The 
prototype product was not completely developed and the 
manufacturing system was still being designed. It was apparent 
that a task analysis of current jobs would not provide the 
information needed to design a valid training program for the new 
production system. 

To initiate the design process for the training system, 
staff ambers in the prototype lab were asked to envision the 



64 

74 



type of manufacturing system required. A focus group format was 
used. Prototype lab staff members were encouraged to be 
creative. Since the product being developed and the technology 
involved was confidential, the prcJject researchers were not able 
to directly participate in this process. However, the company's 
human resource development staff worked with the prototype lab 
staff members. HRD staff members reported that the process 
worked effectively. They were able to develop a vision of the 
type of production system that would be used and the jobs that 
would be needed to operate the system. They also indicated that 
the prototype lab staff members were very excited about being 
involved in the process and continued to provide suggestions 
after their meetings. 

Although the researchers have not had an opportunity to 
tr jk the accuracy of the forecast developed, this approach 
appears to be promising. At the very least it provides a general 
framework from which the specific manufacturing system can be 
designed and developed, and facilitates concurrent training 
program development. Moreover, this approach has the potential 
to provide parallel planning for the development of the 
production system and the total training program required for the 
production system. In addition, it ensures that training an up- 
front consideration throughout the development process. 

Since this envision process is a projection from estimates 
and somewhat sketchy data, it will be important that as the 
product and production systems are refined, the human resource 
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development/training staff must continue to monitor the 
competencies required to effectively operate the evolving 
systems. This information can then be used to refine and modify 
the training programs developed. It can also be used to trigger 
JIT training activities which fill in the gaps in employees' 
competencies. 
Processing the Data 

Data from the envision process can be analyzed by an 
individual or group process. A group process provides an 
opportunity for additional data inputs; and a chance to exchange 
ideas. 

Although there is no system for generating completely 
accurate predictions of the importance or likelihood of new 
technology, there are certain characteristics and conditions 
associated with technologies that are widely accepted as 
indicators of their importance and influence. When current 
technologies have been stretched to their maximum, a new 
technology that has the potential to improve productivity and 
efficiency has a good chance of being adopted. Computers are a 
good example. Computers were initially developed ana have been 
upgraded to help people deal with growing volumes of information. 
Any technology that makes it possible to do things that were not 
possible before, typically has a major impacts. For instance, 
the development of the laser facilitated the development of 
precision cutting, measurement and surgery techniques that were 
not available before, its creation. 
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Some of the indicators of the importance of a new 
technological development and its applications are the amount of 
space devoted to reports and articles on the technology, the pace 
at which the technology is adopted, and the extent to which it is 
applied in new fields. 

Educators will need to determine or estimate the relevance 
of a new technology to their programs. Individuals in business 
or industry need to look at the r, ne technology and determine its 
relevance for their company, it may be helpful to have a small 
group of people discuss the technology and its applications. One 
method, the brainstorming format for example, can be used to 
identify these implications. After the implications have been 
discussed, the group can prioritize them and identify the most 
important ones. An alternative method, the Delphi process, could 
be used to determine priorities. 

At this stage your organization will need to make a decision 
concerning the new technology. The information may indicate that 
the technology has not been sufficiently developed to warrant its 
use at the present time. If this is the case, someone should 
continue to monitor its development if there appears , be some 
potential application, if the information analyzed indicates 
that the new technology has potential value, but more data is 
needed before a decision to implement can be made, you will need 
to actively seek more data. A more active process of contacting 
people who c.re using the technology can be implemented. This 
cc^ld involve contacting researchers, universities and companies 
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experimenting with the technology; and attending technical 
conferences. Another approach would be to purchase the 
technology and experiment with it. 

Other approaches that can be used are simulations and 
scenarios on the applications of the technology. The simulations 
could be done on computer. Another process that can be used 
effectively when the technology involves organizations and 
production systems is role playing. In role playing the 
information collected in previous steps can be vised to define the 
new system, identify the roles of people in the system, and 
describe the outputs of the system. Individuals in your 
organization can then play these roles and determine if the 
system will be effective, efficient and worth adopting. 
Experience gained through the role playing can also be used to 
identify the types of competencies people have to have to manage 
and operate the new system and the types of changes that will be 
necessary in the organization. 

In almost all of these situations it will be difficult to 
totally quantify the results. This is where the small group 
discussion and consensus format is valuable. The group should 
review the data available and consider the outcomes of inaction, 
doing nothing, and the potential outcomes of changing to the new 
technology and/or systems. 

Facilitating Change 
Frequently individuals and organizations are resistant to 
change. A variety of factors may cause this. People may not 
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perceive a need to change and thus do not want to invest extra 
time and effort. In other instances, change may threaten their 
security. Organizations often do not encourage and reinforce 
change, as a result, people are reluctant to change. 

Wnether intentional or not, many organizations are not 
designed to facilitate change. Innovation is not encouraged or 
reinforced. The systems and procedures used in many 
organizations make it difficult and time consuming to change. In 
other organizations communication channels are long and cluttered 
with obstacles. 

Charact eristics of Innovative and Effective Organ isations 

Fortunately, a number of innovative and effective 
organizations exist in our country. Peters and Waterman (1982) 
studied six J . y-two companies that were selected based on their 
high levels profitability and innovativeness. The following 
characteristics were common to these companies. 

1. Action was encouraged. Employees were encouraged to develop 
solutions to problems and design new products rather than 
bouncing a problem from one committee to another or one level to 
another in the company without any result and action. 

2. These companies stayed in close touch with their customers 
and kept informed of their needs and preferences. 

3. Small companies were formed within the corporation to 
encourage independent thinking and to promote entrepreneurship. 

4. All of the companies had a great concern for their people. 
They worked to make them feel that they were a part of the 
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organization and that their contributions were important. 

5. Managers and administrators in effective organizations have 
hands-on experiences with the organization's essential 
activities. 

6. Effective organizations focus on what they know best. 

7. A flat organizational structure is used. There is a focus 
on teamwork and a minimum of administrative levels. 

8. The organizational climate encourages dedication to the 
primary values of the organization, but innovation and deviations 
from these values are also excepted. 

Modeling 

Richard Walton commented that innovation cannot be effective 
unless it is guided by a vision manifest in a model. A model is 
a general concept of the future organization and evolves from an 
understanding of the limitations of traditional organizations and 
experimentation with alternatives (1985, p. 15). Walton noted 
that three processes were important to innovation: (1) process- 
sponsorships, (2) consensus, and (3) information. The first 
process, innovation, requires someone to champion or sponsor the 
concept or new approach. There is a need to establish the fact 
that the organization needs to change. The second, consensus, 
involves gaining the agreement of the various groups and people 
involved in the systems that will be effected by the change. The 
third process, the transfer of information, facilitates the 
development of sponsorship, consensus and construction of an 
effective model. People in an organization need to know what 
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their competitors are doing, the new technologies that are 
available and how new systems car. be implemented. 
Factors Influe ncing Change 

Studies of the change process and organizations that have 
been innovative provide insights on factors that influence 
change. 

1. Information and Awareness. The individuals and units that 
will be influenced by change need to be kept informed of the need 
for change and the potential responses to this need. 

2. Team Work. The individuals who will be involved in the 
change process and affected by it, must be a part of the planning 
process and fully participate in the activities initiated to 
create the desired change. 

3. Effective communication System. Innovation and change are 
facilitated by effective communications throughout the 
organization and across levels in it. 

4. Rewards and Reinforcement. The organization must encourage 
innovation and change through its system of rewards. 

5. Organizational Structure and Procedures. The change 
approval process within an organization must be simple enough to 
permit the survival of innovative ideas and approaches. 

6. Make the change Process Fun and Interesting. People are 
much more likely to participate in activities that they find to 
be interesting and fun. 

Informat ion and Awareness . 

People are reluctant to become involved in an activity or 
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project unless they have sufficient information about it. They 
want to know why the activity is needed, how it will be 
conducted, and the value of the potential outcomes. Several 
information scanning and activities were described previously in 
this section. Involvement in these activities will help to keep 
people informed. They also need to receive adequate information 
on what is happening in their organization. 
Team Work . 

The team work concept builds on the first change principle. 
By being involved, members of an organization will develop a more 
personalized knowledge of the innovation, their role in it, and 
the overall goals. Peters and Waterman commented that "those who 
implement the plans must make the plans (1982, p. 31). 11 Walton 
(1987) recommended that the people who would be effected by 
change should develop the change model. Participating in a team 
also expands individual's perception of the importance of the 
innovation and his/her role in it. They also have a better idea 
of the types of commitments that others are making to the process 
and as a result develop more commitment to the overall plan. 

Effective Communication . 

An effective communication system which involves all levels 
of an organization keeps its members well informed and builds 
trust. An effective communication system also provides timely 
responses to questions and requests that are developed by team 
members. This permits planning and implementation of the change 
process to progress on an orderly schedule. 
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Rewards and Reinforcements . 

Appropriate rewards need to be available to recognize and 
reinforce innovative behavior. This also means that people who 
try new approaches and systems must be able to fail without being 
punished. A wide range of rewards can be used. These range from 
a simple process of recognizing those who have worked or 
participated in an innovative project to more complex reward 
systems that permit people to spin out their won projects or 
departments. First hand access to the latest information, 
opportunities to attend conferences and conventions, the changes 
to expand one's skills and the opportunity to manage one's time 
are also reinforcing. An effective team leader will identify the 
actions and conditions that reward and reinforce team members and 
use these effectively. Before leaving the reward section, it is 
also appropriate to comment that people should not be rewarded 
and recognized for maintaining the status quo if innovation is 
desired. 

Organizational Structure . 

The organizational structure, policies and procedures can 
facilitate or be a deterrent innovation. In one company in which 
the researchers worked, it took more than a year to have a 
product change order processed and approved. Although the 
employees in this unit were actively changing processes and 
characteristics of the product to make it better meet the needs 
of its customers, they seldom if ever filed a change order. It 
took so long to process and the procedure was so difficult that 



73 



they did not feel that it was worth their time. Fortunately the 
company has a relatively stable work force and has developed a 
reputation for high quality products and services. However, the 
product being produces is not quite same one that is described in 
the design specs. People work harder and are more innovative 
when the process is fun. 

During the High-Technology Training Model Project, staff 
members worked with educators in secondary and postsecondary 
school systems and industrial trainers in area companies. These 
experiences identified several potential problem areas when 
different types of institutions worked together. The differences 
in the cultures of the several organizations posed some problems; 
however, the positive outcomes of the interactions greatly 
exceeded the magnitude of the problems encountered. All of the 
participants learned and grew from the experience. Some of the 
major cultural differences are discussed in the following 
paragraphs . 

The philosophies of organizations vary. The *ypes of goals 
and emphasis placed on them are obviously different from 
education to business and industry. However, even across 
educational organizations there is a difference. In addition, 
there are differences in the types of strategies and activities 
that can be used to accomplish these goals. It is important that 
these differences and commonalties be discussed as organizations 
initiate a cooperative venture. Once these have been identified, 
participants will have a better understanding of how each 
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organization operates and where there may be potential conflicts. 

The reward systems in organizations also vary, in a 
company, people are reinforced for developing more efficient 
production systems and producing products that satisfy customers 
needs and interests. University professors are rewarded for 
research conducted and publications. When these two reward 
systems interacted, we found that people in business and industry 
were interested in immediate improvements in their production 
systems and products. They were not concerned about 
documentation and reports. On the other hand, university staff 
members were concerned about adequate research designs, 
appropriate methodology and rigorous research reports, it is 
obvious that people from the two organizations need to clarify 
their needs, interests and approaches. 

Another organizational condition that causes problems are 
the work schedules of organizations, companies work on a year 
around basis with some interlopes in production depending upon 
demand in the consumer sector. When possible, a company would 
like to make its changes during a lull or relatively slow period, 
colleges and universities work around the typical class schedule 
which runs from approximately the first of September through the 
middle of May. There are times during the school year when it is 
difficult for faculty members to be away from their classrooms 
and offices. Also, often faculty members have other pr 'ects and 
activities scheduled for break periods and the summer. The 
scheduling of cooperative projects needs to be careful Iv done to 



take into account the typical schedule of each participant. 

If these differences are recognized up front, it is much 
easier to establish a good working relationship. Time must be 
given to developing an effective working relationship. 
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Needs Assessment 
I ntroduction 

This module discusses the role of the needs assessment 
process in developing and revising courses, programs and 
services. Several needs assessment techniques are listed and 
references given. Procedures for using these techniques are not 
defined since they are readily available in reference books and 
manuals. 

Purpose 

The goal of this module is to describe the general functions 
of and identify alternative ways to conduct a needs assessment. 

Objectives 
The objectives of this module are to: 

1. Define "need". 

2. Describe the role of the needs assessment process in 
developing training and educational programs. 

3. Identify alternative techniques that can be used to complete 
needs assessments. 

Needs Assessment 

Definition of Need 

A need is the difference between what is and what should be 
or what oucrht to be. For example, if technicians can read at the 
eighth grade level but the technical manuals used in their work 
are written at the twelfth grade level, there is a need to 
improve the reading level of these people. The area Ci? need is 
identified by the types of technical literature that has to be 
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read and the extent of need is determined by the difference 
between the technicians' current reading level and the level at 
which they must read. 

A needs assessment is used when one wants to design and 
implement relevant programs and services. The needs assessment 
process provides information on the client's or customer's needs, 
interests, and/or desires. 

Needs can be identified in several ways. One approach is to 
do as described in the example above, contrast the difference 
between the level at which people are performing and the level at 
which they must perl or m. in order to do this you must have a 
clear prescription of the competency level that is required and a 
valid measure of performance in relation to this standard or 
level. The organization may already have a set of standards or 
the envision process described in the envision section can be 
used to establish the level at which people must perform. As 
mentioned in that section, a vision can be established by 
selecting an ex: ing model or creating a new one. An existing 
model can be used when an organization wants to implement a 
proven system or technology. For example, an organization that 
decides to implement a total quality program can select one of 
several programs that have been effectively used by other 
companies and organizations. When an innovative or leading edge 
system is to be implemented, the organization will need to 
develop its own model. In either case, an effective model will 
identify what should be . 
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After the target or required performance level has been 
identified, this level can be compared with the existing level to 
determine the type and extent of need. This process works 
effectively and efficiently in situations where the target levels 
can be specified in quantative terms. In some situations it is 
not possible to do this. Thus, some alternative methods for 
identifying needs are required. 

Another approach to identifying needs is to ask people who 
are involved in the need area to estimate the extent and type of 
need. To illustrate this, consider people who are working in 
their first assignments as teachers. They are involved in a new 
set of experiences and challenges, in most instances, they have 
not been completely prepared for all of the situations they 
encounter, if surveyed, they can provide considerable feedback 
on the types of competencies they need to develop. This process 
has been successfully used in a number of situations by our 
Center to identify the competencies needed and the types of 
education or training required. This process can also be used to 
identify other types of needs, such as equipment, services, and 
facilities. 

Another way to identify needs is to review the information 
on problems and mistakes made in an organization. If problems 
and mistakes are summarized and analyzed, it is possible to 
identify reoccurring patterns, pareto Charts are used by some 
companies to identify the problems that are most common and 
costly. A Pareto Chart is a graphic summary of the frequency 

82 

no 



and/or cost of specific types of problems experienced in an 
organization. The Pareto Chart can be reviewed to identify the 
most common problems. Once this is done, other techniques can be 
used to determine the causes of the problems. In this situation 
the need becomes the actions or the competencies required to 
eliminate the problems. 

In actual practice it may be necessary to use two or more of 
these techniques in combination. For example, the Pareto Chart 
could be used to identify the most critical problem or problems. 
These problems could then be analyzed to isolate the major cause 
or causes. Once this had been accomplished a quantative or 
qualitative analysis could be done to identify specific education 
and training needs. 

Needs Assessment Techniques 
Several needs assessment techniques are presented in Table 
1. The type of needs assessment, its typical applications, and a 
source for further information are included in the table. 
Figure 1 : Techniques Used in Needs Assessment 

Technique/Description Applications References 



A. Techniques for Identifying Problems 



Focus Group 

Group interview process that 
is guided by questions asked 
by the interviewer. 
Interactions between group 
members are an important 
part of the process. 



Very useful for Morgan 
acquiring data on (1988) 
individuals 1 
perceptions , problems 
and needs. 

(Figure cont.) 
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Nominal Group 



Effective process to encourage 
participation in groups 
composed of people with diverse 
backgrounds and experience. 
Easy to use and usually is 
productive. 



Delphi 

Technique utilized to gain 
consensus and develop 
projections of future events. 
A panel of experts is 
surveyed to generate the 
projections. 



Can be used with Moore 
employers, (1987) 
employees, teachers, 
students and/or 
parents to identify 
needs and prioritize 
them. 



Could be used to Nelson 
project what a (1989) 
job or organization 
will be like at some 
point in the future. 
Can also be employed 
to identify and obtain 
consensus on current 
needs as perceived by 
a panel of experts. 



Syst e ms Analysis 

Process used to identify 
the inputs, processes, 
outputs , en v i r onme nt a 1 
factors and the inter- 
actions that are involved 
in accomplishing a goal. 



Used when several 
factors interact 
to influence an 
outcome. 



Halfin and 

Nelson 

(1987) 



B. Techniques for Identifying Causes 
Pa re to Chart 



A Pareto chart graphically 
identifies the relative 
magnitude of the six or 
seven most common problems 
that occur in a product, 
service or organization. 

Cause-Effect Diagram 

"Fishbone Chart" is one type 
Usually the problem effect 
being studied is placed on 
the right-side of the 
diagram. An arrow is drawn 
to the problem. This is the 
cause side of the problem. 



Could be utilized 
to identify the 
most critical 
problem (s) . 



Ishikawa 

(1976, 

42-50) 



The procedure 
facilitates 
breaking a problem 
apart and identify 
ing potential causes. 



Ishikawa 
(1976, 
18-28) 



(Figure cont.) 
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Branches are placed on the 
arrow to indicate potential 
causes . 



C. Techniques for Identifying Tasks and Competencies 



DACUM 

Technique that employs a panel 
of supervisors and employees 
to identify the tasks required 
to perform a job. The panel 
also arranges and sequences 
related tasks in horizontal 
arrays called "bands." 



Task Analysis 

Procedure used to identify 
tasks , their importance , 
and the level of proficiency 
required. Interview and/or 
survey procedures are used 
to gather the data. 



Relatively quick Hal fin 
way to identify Nelson 
the tasks and (1977) 
level of proficiency 
required to do a 
specific job. Works 
effectively when job 
content is relatively 
stable. 



and 



Used to identify 
the tasks and 
competency levels 
required for a 
specific job. A 
survey can be used 
to acquire input from 
a large number of 
people • Works 
effectively when job 
content is relatively 
stable. 



D. Techniques for Identifying Individuals 1 Competencies 
Testing and Assessment Processes 



Zemke and 
Kramlinger 
(1985) 



Performance measures, written 
tests , interest inventories , 
and other data gathering 
techniques used to determine 
knowledge , skill , abil ity , 
interests , goals , etc • 



Use to determine 
the current or 
"what is" level 
of knowledge, 
proficiency, or 
interest. 



Tenopyr 
(1987) 
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S ummary 

As mentioned in the opening section, need is the difference 
between what should be and what is. Needs assessment techniques 
are utilized to identify the what should be . and what is . The 
difference between the what is and what should be identifies the 
type and level of r«2ed. In some situations it is not easy to 
identify the what is and what should be . An alternative form of 
needs assessment requests people to identify where there are 
differences between what is and what should be and how critical 
these needs are. 

Several needs assessment techniques are identified in the 
previous pages. These techniques can be used in a variety of 
settings and situations. No doubt there are other valid 
techniques as well. The researcher or program developer will 
have to identify and select the most effective and efficient 
technique for a given situation. 

In some situations it may be necessary to use more than one 
technique. For example, a Pareto Chart and analysis will 
disclose the most critical problems but will not identify causes. 
To do this, it may be necessary to use a Cause-Effect Diagram, or 
do a systems analysis. If these techniques identify inadequate 
or inappropriate employee competencies as one cause of the 
problem, a task analysis or DACUM process would need to be used. 
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PLANNING AND CONDUCTING A 
STATEWIDE AND REGIONAL SURVEY 
OF BUSINESSES AND INDUSTRIES 

Introduction 

This module describes the procedures and actions required to 
plan and conduct a statewide or regional survey of businesses and 
industries. In some instances, the reader is referred to other 
modules, such as Selecting Appropriate Sampling Techniques that 
are used in planning and conducting the survey. The body of this 
module describes the steps and procedures needed to plan and 
complete a mail survey, in some instances, additional details 
have been included in appendices. Where this is done, the reader 
will be referred to the appropriate appendix. 

Purpose 

The purpose of this module is to provide the information, 
instructions, and procedures required to design and conduct an 
effective statewide or regional survey of businesses and 
industries. 

Objectives 
The objectives of this module are: 

1. Identify the situations in which a survey of businesses and 
industries is an appropriate technique to acquire data. 

2. Provide information and procedures that will allow the 
reader to plan and conduct an effective statewide or regional 
survey of businesses and industries. 

3. Provide information and instructions on how to develop a 
valid survey instrument. 
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4. Describe hew to select a valid sample. 

5. Describe the advantages and disadvantages associated with 
various survey procedures. 

6. Suggest how to use the survey results. 

Value of Statewide and Regional Surveys 
of Business and Industry 

The purpose of a statewide or regional survey of business 
and industry is to obtain pertinent input from employers and 
employees for the improvement of educational programs and 
services. 

When administrators and program directors/managers need 
information for program assessment, evaluation and decision 
making, they usually go to the source. They go out into the 
state or region they service to get information because that is 
the best source for the required information. Although other 
means of going to the source are available, a frequently used 
approach is the use of a mailed survey instrument or 
questionnaire. Surveys are a primary source of information. 
While surveys provide timely information in response to specific 
research questions, they are costly and provide less control. 
The administrator or manager conducting the survey is dependent 
upon the respondent's ability and willingness to provide accurate 
information. 

If there is any intent to infer anything about the 
population from the data derived from the collected survey 
information, appropriate sampling techniques (see sampling 
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module: Selecting Appropria te Samp li ng Techniq ues), must be 
used. 

Planning and Conducti ng a Statewide 
or Regional Survey o f Business and Industries 

The starting point in the development of a statewide or 
regional survey of businesses and industries is to identify the 
objectives for the study. The best way to do this is to 
determine the types of information needed to make the curriculum, 
instructional, and general program decisions being addressed. 
This information can be obtained by asking teachers, 
administrators, school board members, advisory group members, and 
others concerned with your school system what information they 
need to have. This can be accomplished by interviewing samples 
of these people or by using focus-group sessions. The focus- 
group technique is described in the module, Working with Small 
Groups. Another useful technique is to record the information 
needed as you do program and curriculum revisions. 

There are three basic techniques for conducting surveys: 
(l)mail, (2) telephone, and (3) interview. Each of these 
techniques has its inherent strengths and weaknesses as an 
information-gathering process. The criteria for the selection of 
one of these information-gathering techniques over another are 
the type and amount of data desired, and the cost of the 
information-gathering process. This module will be limited to 
mail and telephone surveys. For the purpose of this High- 
Technology Training Model Handbook, interviews will be addressed 
separately since they can serve very specific objectives directed 
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at the interviewers. 

The administrator/manager conducting the survey may wish to 
get hard data or knowledge from the respondents. An example 
might include the number of employees by occupation, products 
produced or services provided, production techniques, and 
specific processes (production and/or information handling) used. 
The statewide survey of businesses and industries conducted as 
part of the High-Technology Training Model Project addressed such 
topics as the types of business, the organizational structure 
(single site or multiple site) , and the number of employees. 
Attitudinal or soft/abstract data is a second type of meaningful 
information that includes the interest, opinion, and feelings of 
the respondents. 

A third kind of data is behavioral, and would include past, 
present, and intended behavior. Again, the High Technology 
Training Model Project's statewide business and industry survey 
sought to get the respondents' opinions as to the annual 
frequency with which specific areas of high-technology oriented 
training were provided and the types cf assistance or help they 
could use in planning, conducting and managing their 
training/ educational programs. 

The final major type of data or information sought in 
surveys, referred to as demographics , includes characteristics of 
the respondents which can be used to tie the collected 
information to the general population. 
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Mail Surveys 

Among the most widely used information-collection 
techniques, mail surveys are relatively low cost and can be self- 
administered. Appendix A is the questionnaire used for the 
statewide survey of businesses and industries conducted as part 
of the High-Technology Training Model Project. 
Advantages 

Although the time required to collect data makes mail 
surveys less desirable for specific applications, they do have 
matched advantages. 

1. Mail surveys provide the respondent a high degree of 
anonymity which may be necessary for the collection of 
potentially embarrassing or sensitive information. 

2. Geographic dispersion and/or the accessibility of the 
information sources may require the use of mailed survey 
questionnaires . 

3. Mail surveys are generally cheaper than telephone or 
personal interviews. 

Questionnaire Construction 

The construction of the survey form or questionnaire is much 
more critical for mail surveys than telephone interviews, once 
printed and mailed, the questionnaire must clearly convey to each 
respondent what each question really means. In a questionnaire 
which provides structured responses (multiple choice or scaled 
ratings) as apposed to space for open-ended responses, the 
provided choices should encompass all possible answers to reflect 
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the respondent's response without ambiguity. 

With no simple rules that assure the development of a good 
questionnaire, Fay and Wallace (1987) identify the following ll 
problem areas frequently associated with mail surveys. 
Consideration and awareness of these problems should help one 
avoid developing or using a problemed questionnaire. 

1. Vocabulary . The vocabulary used must be appropriate for 
the target audience. The emphasis should be simpler, more 
explicit terms as opposed to difficult ones. Acronyms and 
technical terms should be avoided. 

2. Predictable Questions . Avoid asking specifically worded 
questions to which you can predict the respondent's answer or 
which can be answered as Yes or No. Such questions do not 
provide meaningful information. 

3. Unanswerable Questions . Questions the respondents could not 
be expected to have the knowledge to answered. You could ask 
manufacturing firms in general terms the dollar value of their 
annual production of "X" product but it may be inappropriate to 
ask what percentage of the total state production of "X" product 
their firm produces. 

4. Multiple Questions . Don't ask respondents two or more 
questions in a single sentence. Each question asked must be 
limited to one item or concept. 

5. Ambiguous Terminology , if one asked an employer how 
satisfied +"hey are with graduates from their institution the 
question is ambiguous. Is the question referring to their 
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current performance, entry level knowledge, work ethics, specific 
technical skills, or is the question referring to the time 
required for them to reach the required performance levels. 
6 * Sufficient Space. • if respondents are asked to express their 
opinion in response to open-ended questions, it would be 
inappropriate to only give space for a few words. This also 
holds true for multiple choice questions when no space is 
provided after the choice of other ( please specify) . 
7 « Nonbipolar Adjectives. Bipolar adjectives are often used 
when it is desirable to have respondents describe products or 
attitudes; for example "Overall how would you rate the 
satisfaction your job provides? No satisfaction . . . Totally 
satisfying." The problem arises when the adjectives used, which 
are to represent opposite ends of the spectrum are not clearly 
related. If fun was substituted for no satisfaction r the 
respondent can not provide a reliable response. 
8 - Poor Choice Sets for Structured tncwra a variety of 
shortcomings are possible in the answer sets used to develop a 
questionnaire : 

a. Omission of the choice other or don't know is a 
frequent failure. 

b. Inclusive sets must exhaust all possibilities while 
preventing multiple selections. For example; Lf respondents were 
asked to indicate the number of employees with the following sets 
5-20 employees, 21-50 employees, 50-99 employees, and 100 or more 
employees; respondents would not have been allowed to answer less 
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than five employees, and a respondent with exactly 50 employees 
could be in either of two categories (21-50 or 50-99) . 

9. Improper Forced Choice : A frequent approach in 
questionnaire development is to force respondents to choose one 
alternative in a study of job training requirements (i.e., there 
are apprenticeship, associate degree, and certification 
requirements) . A properly developed questionnaire should also 
provide the respondents the probability of identifying a job 
requirement category of None or Other fSpecifvl . 

10. Assumptions about Respondents . Asking "Which of the 
following skills should be stressed for our graduates?" makes 
the assumption that the respondents wants graduates with specific 
skills. A "None" or "Other (Specify)" category should be 
provided . 

11. The Interesting Question . One of the greatest failing of 
those who develop questionnaires is the asking of the interesting 
but unneeded questions. If such demographic question such as the 
age or political preference of the respondent is an essential 
element, it should be asked. Fay and Wallace, (1987) suggest an 
acid test. Layout the intended use to be made for each piece of 
data to be collected. If this can not be easily done, one should 
consider dropping the question. 

Response 

Even though the low cost of mailed surveys is an advantage, 
the response rates are typically low. It is statistically 
important to do everything possible to increase the response rate 
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to preclude the possibility of missing data. We note in passing 
some of the devices used, wit. varying degrees of success in an 
effort to increase the response rate. There are no guarantees 
but one might consider one or all of the following guidelines 
based on Fay and Wallace (1987). 

1 * Preliminary Letter . Use of a preliminary letter stressing 
the importance of the survey, benefits, and detailing intended 
use to which the information will be put. 

2. Sponsorship . Presumably t\e neutrality of the sponsor may 
legitimatize the request for information. 

3. Cover Letter. The questionnaire itself should be 
accompanied by a cover lett-r briefly describing the 
questionnaire and the importance of response. The cover letter 
must contain any administrative instructions and assurances which 
are necessary. lb ms normally included would direct the 
respondent to use the stamped, self-address envelope provided; 
and assurance of anonymity. 

4. Ma il Type . The manner in which the mail is used, the mail 
type, may be a critical element. Are stamps or metered mail to 
be used? will the envelopes have typed addresses or will stick- 
on labels be used? 

5. Rewards . Many market survey experts inclose a small amount 
of cash #.25 to #1.00 with the questionnaires, when the results 
may be of interest to the respondents, such an offering may 
increase response rates. 

6. Anonymity . The sensitivity of the information requested 
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will dilute the irp* tance. Anonymity must be quaranteed unless 
permission is re< vest 3d and obtained to disclose any inf orrnat.* ^n 
on individual respondents. 

7. Format . Neatness, clarity, and ease of completing are 
important. Fay and Wallace point out that content and not form 
seems to be the key factor. 

8. Structure of Content . Generally the key appears to be; 
(a) initially grab interest, (b) initially avoid sensitive or 
difficult questions, and (c)ask for demographics last. 

9. The style of questions is reported to affect response rate 
when the questions come up against respondent bias. 
Follow-Up of Non-Respondents 

Mail Follow-UP . 

One cannot be content with a single mailing to 
respondents. If respondents put the questionnaire aside, a 
reminder in the form of a second letter asking for cooperation is 
usually sufficient. It is possible that the original 
questionnaire was never received. It might be appropriate 
therefore to include a questionnaire if a second follow-up letter 
is necessary. If copies of the questionnaire are included in a 
follow-up mailing, one runs the risk of having some respondents 
sending in two or more copies of the questionnaire. To preclude 
sv.nh a problems, some researchers code questionnaires. The 
procedure used with the survays conducted as part of the High- 
Technology Training Module Projects included the use of different 
box numbers in the return address. While it may be possible to 

99 



109 



identify individual respondents, there was no motivation to do 
so. 

Telephonic Follow-Up of Non-Respondent s . 

If the regular follow-up procedures of non-respondents does 
not net a high enough return rate — usually 70 percent or higher — 
at some point it may be more productive to select a random sample 
of non-respondents and try to contact them by phone. If there is 
a time constraint, or if it will be extremely costly to contact 
these non-respondents by phone, a smaller sample could be used. 
The original survey should provide the base set of questions used 
in the telephone survey. It may be necessary to reduce this set 
of questions in order to make it fit within the length of time 
that is appropriate for a telephone interview (10-15 minutes) . 
If that is the case, select the most critical questions. 
Information from the telephone interviews should be processed 
separately. This will allow you to contrast the results from the 
follow-up of non-respondents with the results from those who 
completed their mail surveys, if the results are similar, you 
can generalize your results to the entire population surveyed. 
If they are significantly different, you will need to adjust the 
outcomes in order to make them reflect the characteristics and 
feelings of the entire population. 

Appendix B is a copy of the questionnaire used for the 
telephone follow-up of non-respondents as part of the High- 
Technology Training Model Project's statewide survey of 
businesses and industries. 
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Telephone Survey 

The popularity of telephone surveys has increased with the 
with the lower relative cost and increased distribution of 
services made available with WATTS lines. 

The interaction afforded between the interviewer and the 
interviewee provide marked advantages. First, the speed with 
which data may be gathered; second, flexibility; third, improved 
response rate for sensitive questions; fourth, the timing and 
sequencing of questions can be controlled, and fifth, 
clarification of responses can be obtained. 
Questionnaire Construction 

The telephone questionnaire, as in the case of the mail 
survey, must be pretested. Any problem that arises during the 
pretest or the actual telephone survey itself, can be used to 
refine the remaining interviews. Such modification should be 
well documented to facilitate their discussion in reporting the 
results . 

conduct of the Survey 

The interviewers must be well trained to conduct the 
interview. The interviewer riust polite, intelligible, and 
neutral to the survey topics, interviewers can be easily 
monitored by supervisors. A number of references are available 
to assist in the preparation and conduct of telephone surveys. 
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Data Analysis 

Editing 

Before analysis of the data received in the survey can 
begin, it must be edited and entered. As Fay and Wallace (1987) 
noted, no matter how well the questionnaire is laid out, or how 
clear the instructions, some unexplainable results will be 
received. Rules must be established in the early planning 
concerning three groups of problem responses; (a) returned 
questionnaires with missing data, or unanswered questions, 
(b) returned questionnaires with obviously incorrect, or 
nonsensical responses, and (c) open-ended responses. A major set 
of rules (or protocols) must be developed which will guide the 
coding of responses if a statistical analysis is planned. 
Entered Data 

The data analysis to be used must be based on the study's 
objectives. In some instances, the study's objectives will 
require comparisons. For example, an objective concerned with 
identifying responses by industry/business type will require a 
breakout by industry. Statistically, this is referred to as 
cross-tabulating. This will require pre-planned coding for data 
entry and processing. Many statistical programs are available 
for data analysis with this capacity. The following two points 
should be considered. 

1. Individuals responsible for data analysis must ensure that 
the available software to be used will have the required 
capacity. 
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2. The ciata analyst should also be asked to jointly review the 
printouts . 
Analysis of Data 

A basic statistics book should be consulted for more indepth 
information on these statistics and to explore a wide variety of 
additional statistics available. One should consider consult 
with a person who specializes in statistical analysis. This 
individual can quickly suggest the appropriate types of 
statistics and statistical analyses that would be appropriate for 
your objectives. The statistical analyses used should be 
understood by your audience and appropriate for the data to be 
analyzed. It may be necessary for you to have more sophisticated 
analyses run to address specific objectives of the survey. 

Appendix C provides a brief explanation of the basic tvpes 
of statistics which may be appropriate for explaining the various 
types of data collected in your study. Statistics explained 
include the mean, median, made, range, and standard deviation. 

Write Report 

Two major factors need to be considered when developing the 
report or reports from the business and industry survey. First, 
what are the characteristics and information needs of the 
audience (s) that will receive the report? Second, what are the 
objectives or purpose for undertaking the business and industry 
survey? 

In most situations, there will be several audiences that 
will have an interest in the survey results. Administrators, 
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teachers, advisory committee members, local students, and state 
and regional educational agencies are typical audiences for a 
business and industry survey. There is probably a core of 
information that will be of interest to most, if not all, of the 
audiences. Also, there will be some pieces of information that 
will have special value to specific groups. Before writing any 
reports, it would be a good idea to look at each of the potential 
audiences and identify their primary interests and the types of 
information they can process most readily. After this is done, a 
specific audience can be selected and the report writing can be 
initiated. 

Many audiences are concerned about the outcomes of the study 
and are not as interested in the details on how the study was 
conducted. For these audiences, report the major outcomes first. 
Place those items of greatest interest first in the report. 
Write in a conversational style. A paragraph or :wo at the end 
of the report can be devoted to some of the major details of how 
the survey was conducted and who was involved in the survey. 

You will probably need to do a more formal report of the 
results also. The formal report should contain the following 
sections: (a) Cover Page, (b) Abstract, (c) Table of Contents, 
(d) Part I - Purpose and Objectives, (e) Part II - Review of 
Previous Research, (f) Part III - Survey Methodology, (g) Part IV 
- Findings (Results) , (h) Part V - Conclusions and 
Recommendations, (i) References and (j) Appendices. 
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Disseminate Bfigalfcg 

Dissemination of the results should be made to groupr and 
individuals who could benefit by learning more about the results 
of the survey. Where possible, presentations should be made to 
interested groups. Visual aids that present key results are also 
helpful. A brief handout that contains the major results 
presented provides an effective supplement to the presentation. 
(See Appendix D for sample visuals) . Be sure to share the 
results with teachers, counselors and administrators. Also, some 
of the results may be relevant to current students. Use your 
creativity to identify dissemination strategies that will 
effectively reach people who need to know about your programs. 
Do not be afraid to try something different. 

Implement Changes 

One of the most effective ways of implementing change is to 
involve the people who will need to change and make changes in 
the process of identifying what modifications need to be made in 
existing courses, programs and services. 

Careful attention should be given to the factors and 
conditions that motivate change. Behaviors that are critical to 
the change process need to be reinforced. For instance, interest 
in changing technology could be reinforced with travel funds to 
visit area industries that are implementing new technology, 
attend conferences that present new technology, and/or visit 
school programs that have developed new curriculums. 

After your staff has had an opportunity to review the 
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results of the business and Industry survey, they should be 
Involved in discussion groups to identify needs and priorities. 
Once this has been completed, they can help establish objectives 
and select strategies to accomplish these objectives. 

Resources are also a 1 important part of change. Many good 
ideas for new programs have been stymied by the lack of resources 
to implement them. Additional funding is helpful; however, 
expanding access to community resources can facilitate change. A 
variety of businesses, industries, and individual talents can be 
tapped with little or no cost. Talk with local business and 
industry people to determine how they will help your school. 

Often the plan for change gets hung up with the barriers. 
It is important to identify the objectives, where you need to be, 
for your programs. After this has been done, brainstorm ways to 
achieve those goals. Do not let the barriers or problems 
interfere with the brainstorming process. They can always be 
factored into your planning later. 
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CVTAE 

CENTER FOR VOCATIONAL. TECHNICAL AN I) AC! m. ' L-O 1 \: A i 'UN 

715-232-1382 



July 27, 1989 



UNIVERSITY 0 1- WISCONSIN 

STOUT 

M f: N O M O N I E WISCONSIN 54751 



Dear Colleague: 

As of this date we have not received your Business/Industry Training 
Survey . Enclosed we have included another copy of this survey, your 
experiences and perceptions will provide valuable input into our 
research project. Please take ten minutes of your time to complete 
the survey and return it to us by August 7, 1989. 

New technology skills and competencies in our work force have created 
a need for improved educational and training programs at all levels. 
Information on the types of training you and your company are 
currently doing and planning for will be very helpful in identifying 
new content for post-secondary and secondary education programs, your 
responses will also help to identify areas in which Wisconsin's 
vocational-technical colleges and universities can help meet your 
training needs. The survey also provides an opportunity for you to 
identify the types of assistance that would be helpful to you in 
providing training for your workers and any training related problems 
your company may be experiencing. 

The results of this survey will be used in developing a training 
system model that focuses on helping companies identify their training 
needs and determining the new competencies which should be included in 
post-secondary and secondary educational programs. 

All responses will be kept anonymous. Responses will be analyzed by 
computer and summary tables prepared for our reports. 

If you would like to have a copy of the results of this survey and 
more information on the High-Tech Project, there is space for you to 
place your name and address at the end of the survey instrument , or 
you may send your name and address in a separate lettar. 

Your participation in this research effort is sincerely appreciated. 

Sincerely yours, 



Orville Nelson, Director 
High-Tech Training Project 
CVTAE, UW-Stout 
218 Applied Arts Building 
Menomon ie , WI 54751 



Tim Mero, Research Associate 
High-Tech Training Project 
CVTAE, UW-Stout 
218 Applied Arts Building 
Menomon ie , * t t 54751 
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Please answer the following questions and statements as they relate to your companies 1 
training activities and needs. 

1-36. A list of training areas or topics is gi?en below. Please review each area and 

indicate how frequently training was provided for your employees during the past two 
years. Circle your answer. Leave column 6 blank at this time. 

FREQUENCY OP TRAINING 

1 ~ No Training - NONE 

2 - No Regular Schedule « NHS 

3 - Annually *> AN* 

4 - Monthly/Quarterly - M/Q 

5 - Continuously » C 



FREQUENCY OF TRAINING 



AREA OF TRAINING 



NONE 
1 



1-2. Quality (Quality Mgmt., SPC, etc) 1 

3-4. Participatory Management (Self Managing Teams) ..1 

5-6. Problem Solving/Decision Making/Thinking Skills 1 

7-8. International Marketing/Global Market Place 1 

9-10. Automated Manufacturing (CAM, Robotics, etc) 1 

11-12. Computer Assisted Design (CAD) 1 

13-14. Electronics 1 

■5-16. Flexible Mfg. Cells/ Group Technology/ JIT/etc 1 

17-18. Other Specific Technical Training 1 

19-20. Basic Literacy (Reading, Math, Communications) 1 

21-22. Train The Trainer 1 

* 

23-24. Career Planning/Human Resource Development 1 

25-26. Human Relations Skills. 1 

27-28. Computer Usage .1 

29-30. Artificial Intelligence 1 

31-32. Automated Office Systems 1 

33-34. Other (List) 1 

35-36. Other (List) 1 
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Please go back and place a check nark (✓) in column 6 after each area/topic in 
which you plan to offer training in the next twelve months.- 
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38-44. What type(s) of help could you use for planning, conducting and managing your 
training/ educational program? Check all that apply. 



(38) . Needs Assessment (Identify Training Needs) 

(39) . Designing And Developing Training Materials 

(40) . Planning Training Courses 

(41) . Identifying Qualified Instructors For Our Training Courses 

(42) . Assistance With Conducting and Managing Training Courses. 

(43) . Evaluating The Effects And Outcomes Of Our Training Courses 

(44) . Other . — 



45. What Other Needs Or Problems Do You Have Related To Training? 



46. Type of Business (Check One) 
(1) - 
W - 



_ Agriculture, Mining, Energy, 

Construction 
__ Broadcasting, Publishing, 
Advertising 

Finance, Insurance, Banking, 
Real Estate 
_ Health, Medical Services 
Hotel, Restaurant 

(6) Manufacturing 

(7) Retail, Wholesale 

(8) Transportation, Utilities, 

Communications 

(9) Other (Please Specify): 



(3) 

(4) 
(5) 



47. If you are a Training Director/Human 
Resources Director or other individual 
designated as responsible for training, 
please check your appropriate title below 
(Only one check): 

(1) Training Director 

(2) Human Resources Director 

(3) Personnel Director 

(4) Other (Please specify): 



(5) 



We do not have anyone designated 
as being responsible for training 



48. Organizational Structure: 

(1) Single location business 

(2) Multiple location business 

(Within WI only) 

(3) Multiple location business 

(Some sites outside of WI) 



49. Number of Employees (WI only): 

(1) 1 " 19 

(2) 20 - 99 

(3) 100 - 499 

(4) 500+ 



If you would like a copy of the results of this survey study, please indicate this and print 
your name and address in the space below: 

Name Title ~ 



Address 



City 



State 



Zip 



Please return the survey in the enclosed envelope. Thank you for your time and input. 

Orville Nelson, Director 
High Tech Training Project 
CVTAE, UW-Stout 
218 Applied Arts Building 
Menomonie, WI 54751 
rn 9^ (715) 232-1362 
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Two copies of a survey were mailed to your business/ firm, one during June and the 
other during August. Since we did not receive your response, may I have a few rinutes 
of your time? 

This survey is part of research being conducted to identify competencies where 
emphasis must be placed in the secondary and postsecondary schools to prepare 
Individuals with the competencies needed by business and industry. 

I have five general questions I would like to ask concerning the training conducted 
by your business or firm. 

1. Was training provided to employees during the past year? 

yes no 

2. In what areas or on what topics was the training provided? 



3. What was the training frequency? 

None No. Reg. Sch. Annual Mon/Qtrly . Continuously 

4. In what area or topics do you plan to offer training within/during the next 
twelve months? 



I have four additional questions concerning your business or firm. 

5. What type of business is it? 

6. What is the appropriate title for the individual designated as responsible for 
training? 



7. Organizational Structure: 

(a) Single location business 

(b) Multiple location business 

(Within WI only) 

(d) Multiple location business 

(Some sites, outside of WI) 

Firm Phone# 



8. Number of Employees (WI only) 





_ i - 


19 


00 - 


_ 20 - 


99 


(c) _ 


_ 100 - 


499 


(<n _ 


500 - 


or more 
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APPENDIX C 
Basic Descriptive Statistics 
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Basic Descriptive statistics 

Me&Q. The mean can be used to identify the typical response 
on a scale that has equal response units. For instance, the 
average number of employees would be an appropriate application 
of the mean. If a likert type rating «;cale format is used to 
evaluate certain or services programs provided by your school 
program, the average response would be a way of determining the 
level of satisfaction with each characteristic. 

Median . Another measure of typical value is the median. 
The median is the point at which one-half of the responses are 
above and one-half of the responses are below, it can be used in 
situations where the response intervals are not of equal value. 
For example, the median level of education * employees would be 
an appropriate application of this static : :» when the survey 
question asks for the highest degree or level completed. 

Mode . The mode is the whird statistic used to reflect 
typical value. It is the response choice selected most 
frequently. The mode is useful in situations where the responses 
to a question are categorical and are on a continuum or scale. 
For instance, the mode would be the appropriate statistical 
measure for reporting the city, county, region, or area in which 
most of your responding businesses or industries are located. 

Range . In some instances it is desirable to report the 
variability in the responses as well. The range is probably the 
most commonly used measure of variability. The range of the 
number of employees for example, may be as interesting as the 
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average annual sales. People hear and see the ranges in 
temperatures each day as they view the weather news on TV. They 
also have an opportunity to experience the variations in 
temperature during the day and thus have a sense of what this 
variability means. The range is the difference between the 
largest and the smallest values. To illustrate this, suppose 
that the number of employees reported had a high value of 5,400 
and low value of six employees, with a range in the number of 
employees of 5,394, the indication is a wide range in the number 
of employees for the businesses and industries in the sample. 

Stand Deviation. Another measure of variability, the 
standard deviation, is an indicator of the degree to which 
individual values vary from the mean of the total set of values. 
The standard deviation has a more complex formula than any of the 
statistics mentioned so far. Typically it would be calculated by 
computer. A standard deviation of zero indicates that there is 
no variability, in other words, that all of the values are the 
same. As values spread out from the mean, standard deviation 
will become larger. 
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* Agriculture, Mining, Energy, Construction 9% 



■+ Broadcasting, Publishing, Advertising 0% 

"* Finance, Insurance, Banking, Real Estate 1 % 

"»+ Health, Medical Services 4% 

Hotel, Restaurant 0% 

Manufacturing 76% 

,B * Retail, Wholesale 6% 

■»* Transportation, Utilities, Communication 0% 

Other 4% 
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B/l Statewide Training Survey 



Demo'graphicsi Number of Employees 

•*"\ ____ . 



Number of Employees (Wl only) Fall 1 989] 



10 




B/l Statewide Training Survey 



Demographics: Organizational Structure 



Fall 1989 



Single Location Business 



Multiple Locations (Some outside Wl) 



Multiple Locations (Within Wl) 



Omits H2°/c 



54% 




0% 10%20%30%40%50%60% 




Sample of Non-respondents (nn=35) 
Respondents (n=67) 
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B/l Statewide Training Survey 



...... 



High-Technology Related Training 

Survey # 



n = 67 Return 24% 



Fall 1989 



Frequency of training they provided to employees during the 
last two years, and training projected for the next 1 2 months 




Dyrioq Past Tw© Yean 



Tvne of Trainind 

X Y V/ vl X A C4AAA1A1.C 


No 
Training 


No 
Regular 
Training 


Annually 


Monthly/ 
Quarterly 


Contin- 
uously 


During 
Next 12 
Months 


Quality (Quality Mgmt., SPC, etc.) 


13% 


33% 


7% 


6% 


11111$$ 


45% 


Particapatory Management (Self Managing Teams) 


1 39% 


25% 


10% 


4% 


15% 


30% 


Problem Solving/Decision Making/Thinking Skills 


31% 


24% 


9% 


13% 


18% 


37% 


International Marketing/Global Market Place 




4% 






a rn 

4% 


3% 


Automated Manufacturing fCAM Robotics etc ^ 


57% 


27% 




9% 


4% 


19% 


Computer Aided Design (CAD) 


, 58% 


27% 




4% 


4% 


18% 


Electronic! 




24% 


4% 




7% 


16% 


Flexible Mfg. Cells/Group Technology /JIT/etc 


Iliil 


19% 


3% 


1% 


7% 


19% 


Other Specfic Technical Training 


28% 


lp36% 


3% 


'•% 


21% 


28% 


Basic Literacy (Reading, Math, Comminications) 


^ ; 70% 


16% 




4% 


3% 


12% 


Train The Trainei 


58% 


19% 


6% 


6% 


4% 


18% 


Career Planning/Human Resource Development 




31% 


1% 


6% 


4% 


15% 


Human Relations Skills 


27% 


46% 


*9% 


4% 


10% 


36% 


Computer Usage 


16% 


43% 


3% 


4% 


27% 


37% 


Artifical Intelligence 


81% 


7% 


1% 








1 g Automated Office System: 


54% 


25% 




3% 


9% 


10% 
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B/l Statewide'Training Survey 

I - — - . ' 



Fall 1989 



No One 



Department Head, Supervisor 



Training Director 



Personnel Director 



55% 



Omits 




0% 10% 20% 30% 40% 50% 60% 




Sample of Non-respondents (nn=35) 
Respondents (n=67) 



17 132 
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B/l Statewide Training Survey 

— — 

Interest in External Training Assistance 



n = 67 Return 24% Fall 1 989 



Types of help respondents would use for 
planning, conducting, and managing 
Training/Education Program. 



Designing & Developing 
Training Materials 

Planning Training Courses 

Needs Assessment 
(Identify Training Needs) 

Identifying Qualified Instructors 
for Training Courses 
Evaluating the Effects & Outcomes 
of Our Training Courses 




Assistance With Conducting and 
Managing Training Courses 




0% 10% 20% 30% 40% 50% 
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HIGH-TECHNOLOGY TRAINING 
MODEL MODULE 



Planning and Conducting an Interview 
Survey of Representatives of/ 
Businesses and Industries 



Developed as a part of the High-Technology 
Training Model for Rural Based Business and 
Industry, Technical colleges and Local and State 
Educational Agencies under Grant No. V199A90151. 



Prepared by: 
J. Timothy Mero 
Center for Vocational, Technical and 
Adult Education, University of Wisconsin-Stout 
Menomonie, WI 54751 
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PLANNING AND CONDUCTING AN 
INTERVIEW SURVEY OF 
REPRESENTATIVES OF BUSINESSES AND INDUSTRIES 

Introduction 

This module describes the procedures and actions required to 
plan and conduct an interview survey of representatives of 
businesses and industries. In some instances, the reader is 
referred to other modules that should be used in planning and 
conducting an interview. The body of this module describes the 
procedures and considerations needed to plan and complete a 
interview survey, in some instances, examples have been included 
in appendices. Where this is done, the reader will be referred 
to the appropriate appendix. 

Purpose 

The purpose of this module is to provide the information, 
procedures and considerations required to design and conduct an 
effective interview survey of representatives of businesses and 
industries. 

Obi ectives 
The objectives of this module are: 

1. Identify the situations in which an interview survey of 
representatives of businesses and industries is an appropriate 
technique to acquire data. 

2. Provide procedures and considerations that will allow the 
reader to plan and conduct an effective interview survey of 
representatives of businesses and industries. 

3. Provide information and instructions on how to develop a 

126 



valid interview survey instrument. 

4. Describe the advantages and disadvantages associated with 
interview survey procedures. 

5. Suggest how to use the survey results. 

Value of Interview Surveys of 
Re presentatives of Business and Industry 

From an educational perspective, the purpose of a interview 
survey o* representatives of business and industry is to obtain 
pertinent input from employers and employees for the improvement 
of educational programs and services. 

When administrators and program directors/managers need 
information for program ar\<* ssment . evaluation and decision 
making, they usually go to the source. One procedure frequently 
used is to survey representatives of businesses and industries to 
obtain information because they are the best available source for 
the required information. Surveys frequently serve as the 
primary means of gathering the required information. While 
surveys can provide timely information in response to specific 
research questions, they can also be costly and frequently 
provide less control. In each of the survey techniques, the 
administrator or manager conducting the survey is dependent upon 
the respondents ability and willingness to provide accurate 
information. Regardless of the type of survey used, the basic 
purpose is to develop the necessary information upon which to 
base decisions. 

The interview survey is the survey technique of choice if 
the gathering of information will be enhanced by its unique 
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characteristics. Fay and Wallace (1987) point out that managers 
make more use of personal interviews as a survey techniques than 
is commonly thought. According to Fay and Wallace (1987) , the 
interview survey technique is well suited to handle: (a) lengthy 
a id/or complex subjects, and (b) were the data received might 
suggest the need to pursue a more detailed, indepth line of 
questioning. The interaction which takes place during the 
conduct of the interview along with the interviewer's ability to 
respond to various stimuli frequently results in a richer set of 
data than can be obtained using other survey techniques. 

The interview surveys of representatives of local businesses 
and industries conducted by the educators participating in the 
High-Technology Training Model Project was considered to be a key 
element in the overall success of that project. Building on the 
unique advantages of the interview survey technique, the 
interview surveys afforded the opportunity and impetus for the 
participating educators to: (a) become more familiar with the 
extent to which local businesses and industries are adopting and 
applying new, high-technologies, (b) assess the applicability of 
their respective curriculums, and (c) identify changing skill and 
competency requirements relative to new, high-technologies. 

The interview survey of local businesses and industries by 
the participating educators had specific value. First, it 
facilitated the movement of the participants out of their 
familiar, and very comfortable classroom, and out into the 
unfamiliar, ever-changing world of work. Second, the required 
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interviews encourage articulation between educators and 
representatives from local businesses and industries. It is one 
thine, to educate students and provide them a laundry list of 
competencies from which they c»n draw at some future date. It is 
quite a different thing for educators to have first hand 
knowledge and be able to explain how the particular competencies 
are being applied. Third, the development of an awareness among 
the participants of the new, or high-technology in use within 
their local area. Fourth, the identification of the new 
technology applications anticipated over the next three to five 
years. Fifth, the identification of the new skills or 
competencies employees will need, now and in the near future in 
response to the adoption and application of new, high- 
technologies. Sixth, the identification of additional entry- 
level skills graduates from the interviewer's school will need. 
The intent was to ha?< the participating educators identify 
additional competency needs for the purpose of curriculum 
development. For the purposes of the High-Technology Training 
Model Project, no attempt was made to differentiate between the 
skills identified by secondary and postsecondary educators as 
interviewers. The intent was to identify the skills and 
competencies without attempting to discern at which educational 
level they should be taught. The final value was determining the 
level of involvement of local businesses and industries, and 
schools. 

If there is any intent to infer anything about the local 
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business and industries based on the interview survey sample 

population and the collected set of data, one must ensure that 

the requisite sampling techniques are employed. The module 

entitled: Selecting Appropriate Sampling Techniques and 

Determin ing Valid Sample Sizes , provides a general discussion of 

sampling techniques considered appropriate for the analysis of 

the accumulated data for the purpose of up-dating curriculum. 

Planning and Conducting an Intervi ew 
Survey of Representative of Local 
Businesses and Industries 

There are three basic techniques for conducting surveys.; 
(1) mail, (2) telephone, and (3) interview. Each of these 
techniques has its inherent strengths and weaknesses as an 
information-gathering process. Criteria for the selection of one 
of these information-gathering techniques over another are the 
type and amount of data desired, the availability of a target 
population, and the cost of the information-gathering process. 

This module will be limited to interview surveys, some time 
referred to as one-on-one or personal interviews . For the 
purpose of this High-Technology Training Model Handbook, mail and 
telephone survey are addressed in the module titled: Planning 
and Con ducting a Statewide or Regional Survey of Business and 
Industries . 

The adm.ii istrator/manager responsible for the coordination 
or conduct of the survey must establish the goal(s) for the 
interview survey. The survey's purpose and objectives will be 
based on the established goals and form the corner stone for the 

130 
1 1(1 



surveys design, development, conduct, and the analysis of the 
accumulated data. 

In the case of interview surveys, there are two distinct 
goals the survey can serve. First, and most obvious, as noted by 
Fay and Wallace (1987), is the development of the needed data 
which may serve as the basis for decision making. Second, and 
less obvious, is the experience and exposure provided to the 
interviewer (s) who has (have) interaction with the respondents 
and other persons or stimuli during the conduct of the interview. 
This is especially true when the interviews are conducted on 
site, at the business or industry. 

In the case of the High-Technology Training Model Project, 
the 103 interviews of representatives of local business and 
industry were conducted by the 45 participating educators. The 
interview survey was a key element in achieving the project's 
goal of bringing business and industry, and education together to 
identify new competencies and skills needed by entry-level 
employees. The goal was not to generate or conduct occupation 
task analysis. The following five, broad objectives were 
established for that interview survey of representatives of local 
businesses and industries. 

1. Get the participating educators out of the classroom, their 
academic environment and conducting on-site visits of local 
business and industries. 

2. Encourage articulation between educators and representative 
of local businesses and industries. 
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3. Develop an awareness, on the part of the participating 
educators, of the new or high-technology currently in use by 
local businesses and industries. 

4. Identification of the new technology application that local 
businesses and industries anticipate adopting within the next 
three to five years. 

5. Identification of the changing competency and skill 
requirements associated with the high technologies for current 
employees and new entry-level employees. 

Advantages 

Apart from the high cost of transportation, which may be 
geographically restricting and preclude using a widely dispersed 
sample, interview surveys have many advantages. According to 
Jaeger (1988), some survey researches consider the selection of 
any of the alternative survey methods as being totally 
unacceptable. The following advantages of the interview survey 
are frequently noted. 

1. The interviewer can provide clarification or any survey 
question the respondent does not understand or misinterprets. 

2. The interviewer has the opportunity to ask additional 
questions to clarify a respondent's answer (Jaeger 1988). 

3. The interviewer can respond to answers — data — received by 
probing with a further line of questioning (Jaeger, 1988; Fay and 
Wallace, 1987). 

4. The interviewer has visual contact with the respondent. 

5. The interviewer can observe the respondent's environment 
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which can be an important factor (Jaeger, 1988) . 

6. The interviewer, if properly trained, can react to the 
respondent's body language which may indicate the respondent's 
willingness to respond, understanding of the question, and if the 
respondent actually has more to offer (Jaeger, 1988) . 

7. It is possible to have much less structure in an interview 
survey (Fay and Wallace, 1987) . 

8. The interviewer can incorporate visual aids, samples and 
other materials to elicit responses from the respondent (Jaeger, 
1988) . 

9. The total interaction which take place between the 
interviewer, respondent, and other persons or stimuli ^an result 
in a much richer set of data. 

10. The rate of respondent cooperation is generally much higher 
with interview surveys than other survey techniques (Jaeger 
1988) . 

11. The interview provides the interviewer with the opportunity 
to assess the respondents environment as a response factor. 
Interview Guide 

Survey developers agree that in order to obtain 
standardized, comparable data fr^a each respondent and each 
interviewer; each interviewer must have a written interview guide 
containing the questions. The guide must indicate "the questions 
to be asked and ir what order, and what additional prompting or 
probing is permitted" (Gay, 1981, p. 166). 
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1. Each question must be clear and unambiguous (Jaeger, 1987; 
Gay, 1981) . 

2. Gay (1981) points out that although ir. -erview surveys are 
generally less structured and therefore involve less structured 
questions. The use of a semi structured question format is 
usually appropriate. In a semistructured question format, the 
interviewer asks a structured question, normally limiting the 
interviewee's response; then follows with a clarifying 
unstructured, or open-ended question. Gay (1981) suggests that 
the interviewee's response to unstructured questions provide 
explanation and understanding for their response to the 
corresponding restricted question. 

3. Each question must relate to a specific objective of the 
study (Jaeger 1988; Gay, 1981). 

4. Interviews should be just as brief as possible (nay, 1981). 

5. The use of leading questions should be avoided (Gay, 1981). 

6. Keep the vocabulary level in the questionnaire appropriate 
for your sample population and as low as possible without being 
insulting (Jaeger, 1988). 

7. Avoid the use of specialized jargon and acronyms, if there 
is no alternative, ensure that such terms are clarified (Jaeger, 
1988) . 

Appendix A contains a copy of the questionnaire developed 
for interview of representatives of local businesses and 
industries by the educators participating in the High-Technology 
Training Model Project. One must keep in mind that the purpose 
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of that interview survey was as much for the edui ation and 
development of the educators conducting the interview, as it was 
for the accumulation of the needed data. 
Conduct of the Interview 

Gay (1981) clearly points out the importance of properly 
training each interviewer to conduct the interview and the impact 
such training can have on producing standardized results. 
Effective communication, therefore, is a critical element. 

The interview must start on the right foot. Before the 
first formal question is asked, sufficient time must be spent 
establishing the necessary rapport, explaining the purpose of the 
interview and study, and assuring the strict confidentiality of 
all responses. 

As the interview continues the interviewer must remain alert 
and let the respondent's reactions guide the interview. 
Recording the Data 

Interviews lend themselves to mechaniceil recording with the 
use of any number of audio recording devices or manual recording 
by the interviewers. If the interviewer is to manually record 
responses, the interview guide must provide adequate space after 
each question. The responses may be recorded as the interview 
progresses or shortly after the interview is completed. Although 
there are advantages to each recording method, the use of a 
mechanical recorder is recommended whenever possible. Mechanical 
recordings are considered to be more efficient and more 
objective. 
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Pretesting 

While Fay and Wallace (1987) indicate that "personal 
interviews may require less pretesting" (p. 131), Gay (1981) 
recommends that the. interview guide, interview procedures, and 
the analysis procedures all be tried out before the actual study 
begins; and that the pretest sample be from or representative of 
the survey population. 

In the case of the High-Technology Training Model project, 
four staff members participated in not less than seven interviews 
to pretest all elements of the interview survey. The staff 
trained tuc 43 participating educators who in turn conducted the 
.103 interviews which comprised the survey. It should also be 
noted that over one-third of the interviews were conducted as 
team interviews by two to three interviewers. In those 
situations one interviewer functioned as the primary interviewer 
actually conducting the interview while the other interview team 
members functioned as observers and recorders. 

Data Analysis 

Edi tin g 

Before analysis of the data received in the survey can 
begin, it must be edited and entered. As Fay and Wallace (1987) 
noted, no matter how well the interview guide and the questions 
are laid out, or how well the interviewer (s) is trained, some 
unexplainable results will be received. Jaeger (1988) as well as 
Fay and Wallace (1987) point out the necessity for establishing 
detailed decision rules (or protocol) for the interpretation and 
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coding of responses. This is particularly important in the case 
of the unstructured, open-ended questions. The key problem as 
Fay and Wallace point out is "false or misleading information" 
(1987, p. 131). Jaeger (1988) stresses the importance of being 
able to assign each response to one (and only one) category. The 
decision rules must insure that all categories are exhaustive and 
mutually exclusive. This may necessitate including such 
categories as miscellaneous and does not apply . 
Entered Data 

The data analysis to be used must be based on the study's 
objectives, in some instances, the study's objectives will 
require comparisons. For example, an objective concerned with 
identifying responses by industry/business type will require a 
breakout by industry. Statistically, this is referred to as 
cross-tabulating. This will require pre-planned coding for data 
entry and processing. Many statistical programs are available 
for data analysis with this capacity. The following two points 
should be considered. 

1. Individuals responsible for data analysis must ensure that 
the available software to be used will have the required 
capacity. 

2. The data analyst should also be asked to jointly review the 
printouts. 

Analysis of Data 

A basic statistics book should be consulted for more indepth 
information on these statistics and to explore a wide variety of 
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additional statistics available. One should consider consult 
with a person who specializes in statistical analysis. This 
individual can quickly suggest the appropriate types of 
statistics and statistical analyses that would be appropriate for 
your objectives. The statistical analyses used should be 
understood by your audience and appropriate for the data to be 
analyzed. It may be necessary for you to have more sophisticated 
analyses run to address specific objectives of the survey. 

Write Report 

Two major factors need to be considered when developing the 
report or reports from the business and industry survey. First, 
what are the characteristics and information needs of the 
audience (s) that will receive the report? Second, what are the 
objectives or purposes for undertaking the survey of 
representatives of local business and industry survey? 

In most situations, there will be several audiences that 
will have an interest in the survey results. Administrators, 
teachers, advisory committee members, local students, and state 
and regional educational agencies are typical audiences for a 
business and industry survey. There is probably a core of 
information that will be of interest to most, if not all, of the 
audiences. Also, there will be some pieces of information that 
will have special value to specific groups. Before writing any 
repoi-ts, it would be a good idea to look at each of the potential 
audiences and identify their primary interests and the types of 
information they can process most readily. After this is done, a 
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specific audience can be selected and the report writing can be 
initiated. 

Many audiences are concerned about the outcomes of the study 
and are not as interested in the details on how the study was 
conducted. For these audiences, report the major outcomes first. 
Place those items of greatest interest first the report. 
Write in a conversational style. A paragraph or two at the end 
of the report can be devoted to some of the major details of how 
the survey was conducted and who was involved in the survey. 

You will probably need to do a more formal report of the 
results also. The formal report should contain the following 
sections: (a) Cover page, (b) Abstract, (c) Table of Contents, 
(d) Part I - Purpose and Objectives, (e) Part II - Review of 
Previous Research, (f) Part III - Survey Methodology, (g) part IV 
- Findings (Results) , (h) Part V - Conclusions and 
Recommendations, (i) References and (j) Appendices. 

Disseminate Results 

Dissemination of the results should be made to groups and 
individuals who could benefit by learning more about the results 
of the survey, where j - .ble, presentations should be made to 
interested groups. Visual aids that present key results are 
also helpful. A brief handout that contains the major results 
presented provides and effective supplement to the presentation. 
Be sure to share the results with teachers, counselors and 
administrators. Also, some of the results may be relevant to 
current students. Use your creativity to identify dissemination 
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strategies that will effectively reach people who need to know 
about your programs. Do not be afraid to try something 
different. 

Appendix B is presented as an example of reporting of 
significant findings u.id conclusions from an interview survey of 
representatives of local businesses and industries. The 
interview survey was conducted as part of the High-Technology 
Training Model Project, and the data was collected using the 
interview guide contained in Appendix A. 

Appendix C contains copies of selected briefing charts 
developed based on the analysis of data resulting from the High- 
Technology Training Model Projects interview survey of 
representatives of local businesses and industries. The briefing 
charts are presented as examples and include: (1) the corporate 
distribution of the businesses and industries surveyed, (2) the 
corporate size, (3) new technologies currently used and/or 
planned during the next 3-5 years (only technologies with a 
combined use/need of 15% or more are shown), (4) new skills and 
competencies current and/or entry level employees need, (5) 
anticipated sources for new technology related skills training, 
(6) current or willingness of local businesses and industries to 
work with local schools, and (7) the willingness of local 
businesses and industries to provide intern experiences for 
teachers . 
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Implement Changes 

One of the most effective ways of implementing change is to 
involve the people who will need to change and make changes in 
the process of identifying what modifications need to be made in 
existing courses, programs and services, it was for that 
specific reason that all 45 of the educators participating in the 
High -Technology Training Model Project were required to 
participate as an interviewer in the interview survey of 
representatives of local businesses and industries. There was no 
outside source who came into their classrooms with a new bag of 
tricks and all the answers. The resulting data is theirs. By 
involving the educators in the development of the data, they 
willingly accepted the results, and were able to implement 
curriculum changes in their courses based on that data. 

After your staff has had an opportunity to review the 
results of the business and industry, interview survey of 
representatives, they should be involved in discussion groups to 
identify needs and priorities, once this has been completed, 
they can help establish objectives and select strategies to 
accompl\t>ri these objective. 

Often the plan for change gets hung up with the barriers. 
It is important to identify the objectives and where your program 
need to be. After this has been done, brainstorm ways to achieve 
those goals. Do not let the barriers or problems interfer- with 
the brainstorming process. They can always be factored into your 
planning later. 
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HIGH TECHNOLOGY TRAINING MODEL PROJECT 



CVTAE 
3/2/89 



B/I INTERVIEW SURVEY 



Directions : Interview three to five people in local businesses or industries 
If possible, select B/l's that employ your graduates. Record the major 
comments on this form. You may want to ask other questions as well. 



Business/ Indus try ______ Date — — 

Address ?itv _ZiP. 



Person(s) Interviewed: 
Interviewer: 



1. What are your major products/services? (If you already know, then go 
question 2) . 



2. How many people do you employ in this city or county? _ 

3. What new technology are you using in your company? (New technology - 
equipment, processes, management techniques, etc.) 



4. What new technology are you planning to use in the next 3-5 years? 



5. What new skills or competencies do your employees need? 



6. How are these skills/competencies developed? (ie. the skills listed in 
4). 
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B/I Interview Survey 



7. What additional skills do graduates from our school need as they start 
work with your company? (If possible, refer to a specific graduate or 
graduates from your school who have worked at this company). 



8. Ask how they work with local schools. Some examples are given below: 

a. Serve on advisory committees 

b. Teach a course(s) 

c. Provide visiting speakers 

d. Provide cooperative education work sites 

e. Provide work experiences for students 

f . Provide 3uauner work experience for teachers 

g. Provide shadowing experiences for students 
h- Host student tours/field trips 

i. Other 

9. Xf they do not do any of the activities listed in eight, ask if they 
would be interested in participating in one or more of these. 



10. Would your company be willing to serve as an intern site for a 

teacher this summer? (ie. Provide paid work experiences or opportunities 
to shadow/observe in the company). 
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APPENDIX B 

Reported Findings and Conclusions of the 
High-Technology Training Model Projects' 
Interview Survey of Representatives of 
Local Business and Industries 
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Reported Significant Finding s and Conclusion s 
for the Interview Su rvey of Representative 
of Loc al Business and Industries 

Significant Findings 

1. In rural West Central Wisconsin, 60.2% of the businesses and 
industries surveyed manufactured a product using raw stock 
material and/or assembled components to produce a product. 

2. Within rural West Central Wisconsin, 77% of the businesses 
and industries surveyed were small business. A small 
business is statistically defined based on the 10th Annual 
Grant Thornton Manufacturing Climate Study, which classifies 
a small business as one with less than 500 employees. 

3. Based on the cumulative response to questions 3 and 4, 
respondents reported the cumulative results for new 
technologies currently in use or planned to use in the next 
3-5 years. 

New Current Planned Combined 

Technology Use(%) Usem Use(%) 

Production 

CAD/CAE 40 3i 71 

CAM 22 23 55 

NC 9 11 20 

CNC 36 20 56 

Robotics/machine centers 28 21 

Jigs & fixtures 16 



49 

21 37 



Flexible manufacturing 5 21 26 

(cell/system) 

Laser cutting & control 18 18 36 
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Alternative materials 15 12 
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New 
Technology 



Current 
Use(%l 



Planned 
Usem 



Combined 
Usem 



Oual itv 

Lasers (measurement) 7 8 15 

Production/product testing 16 18 34 

MRP II 6 8 15 

Electronics 10 5 15 

Material Control 

JIT 16 G 24 

Optical scanning/coding 27 19 46 

Distribution 11 11 22 

Computerized inventory 13 10 23 

Business 

Word processing 15 6 21 

Information storage/retrieval 15 8 23 
Communications 

Electronic mail/FAX 15 15 30 

Telecommunication 16 3.0 26 

Data link (networking) 16 13 29 
Computer Microprocessor 

Applications 35 21 56 

Scheduling (personnel/product) 12 3 15 

Personal computers 19 3 22 

Billing 14 2 16 

Accounting 7 8 15 

On-line applications 13 8 21 
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New 
Technology 

Personal computers- 
Billing 
Account i rig 

On-line applications 
''a ne e Tec hnolog y 



Current 
U sef %) 

19 
14 
7 
13 
11 



Planned 
Used) 



3 
2 
8 
8 
8 



\ 



Combined 
Use(fc) 

22 
16 
15 
21 
19 



4. rased on the combined responses to questions 5 and 7, the 
respondent,-; **£,jrt«ri the cumulative need for new skills or 
coi t^cancicjs current employees and new entry level employees 



i4 -cid * 



Employees 



Additional Skills/ 
Cjomp L ete^cjQ§ 

General 

Publ ished/special 
designed software 

Manage array of 
activities 



Current 



New Entry 
Level 



7 
38 
20 



Control machine software 19 



Programming 
Data management 

.Computer 

Keyboard inq 



11 
16 

40 
16 



27 

9 

0 
6 
0 
0 



Combined 
New Skill 
Requirements 
* 



34 

47 

20 
25 
11 
10 

40 
16 
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Additional Skills/ 
Competencies 

Math 

Applied 

Basic 

Advanced 
Keyboardina 



Current 

% 



21 
16 
7 
16 



New Entry 
Level 



Thinking S k ills/Problem Solving 

16 



Critical chinking/ 
reasoning 



Troubleshooting/problem 

solving 27 

Communications 

General 8 

Oral 19 

Visual/reading 26 

Writing 9 

Technical 

Literacy 27 

Proficiency 48 

Applications 0 

New manufacturing 

processes 0 

Mechanical background 0 

Techniques 0 

Certification 1 



29 
27 
7 



25 
43 

13 
31 
27 
30 

23 
26 
16 

15 
21 
14 
0 



Combined 
New Skill 
Requirements 

% 



50 
43 
14 
16 



41 
70 

21 
50 
53 
39 

50 
74 
16 

15 
21 
14 
1 
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Additional Skills/ 
Competencies 

Work Ethics 

General 

Attendance/punctual ity 
Cooperation 
Self management 
Time management 
Developmental 
Attitudes 
Common sense 
Creativity 
Comprehens ion 
Dexterity 

Follow instructions 
Inguisitiveness 
Interviewing 
Know how to learn 
Physical strength 
Resume writing 
Spatial 

Self confidence 
Willingness to learn 
Adaptation to change 
Self motivation 
Electronics 



Combined 
New Entry Egw skill 



Current Level Requirements 
% % % 

0 10 10 

0 9 9 

2 5 7 

0 14 14 

0 16 16 

15 26 41 

2 6 8 

0 13 13 

0 3 3 

0 4 4 

0 10 10 

0 6 6 

0 2 2 

10 14 24 

0 2 2 

0 1 i 

0 14 14 

0 14 14 

0 9 9 

15 0 15 

8 0 8 

42 12 54 
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When asked how the new skills and competencies current 
employees need are developed, respondents indicated the 



following major sources: 

Source % 

a. Technical Colleges 64 

b. In-plant (informal) 63 

c. In-plant (formal) 62 

d. Vender/Manufacturer 27 

e. University (Scheduled Courses) 24 

f. Secondary School 21 

g. Prior Experience 18 

h. Tuition Reimbursement 16 

(source not specified) 

i. Seminars 12 
j. Workshops, Shows, Conferences 9 



k. Manuals 9 

1. Corporate Out Of Plant 6 

m. Consultants 6 

n. Apprenticeships 6 

o. Professional Organizations 2 

p. Customized Training 2 

5. Respondents indicated positive willingness to work with 
local schools by (1) providing tours - 78%, (2) serving 
on advisory committees - 69%, (3) providing speakers - 
65%, (4) providing student work experiences - 62%, (5) 
providing a Co-op educational work site - 41%, (6) 
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providing a shadow experience for students - 36%, (7) 
providing n summer work experience for teachers - 35%, 
(8) teaching a course - 34%. 

Conclusions 

1. The responding businesses and industries did not downgrade 
or de-emphasize the current curriculum efforts and requirements. 
The findings simply identify additional skill and competencies 
which should be emphasized to provide graduates with the high- 
technology related skills and competencies necessary for 
successful employment in a changing, technology driven work 
environment. 

2. The finding emphasized those skill and competencies 
associated with computer applications and computer aided 
processes to include: design/ engineering, manufacturing, 
material control, quality assurance, and business applications. 

3. The additional skills and competencies needed for current 
employees centered around specific technological applications and 
current production proficiency. The additional skills and 
competencies identified for new entry-level employees reflect 
general skills, production technology, attitudes and ethics. 

4. Electronics (as a skill or competency), like computers, 
appears to be a generalizable area of emphasis which crosses most 
disciplines associated with new, high-technologies. 

5. The role of the technical college as a training source for 
rural-based industries can not be over emphasized. The local 
technical college can directly or contractually provide: (a) 
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formal and informal in-plant training, and vender/manufacturer 
certified training. 

6. The responding businesses and industries, when asked , appear 
to be very willing to work with local schools. 

7. The need for meaningful articulation programs between 
businesses and industries, secondary, and postsecondary schools 
should receive emphasis with each participant serving as an 
equal . 
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APPENDIX C 
Briefing Charts 
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(N= 1 03 Survey Period May 1 989 - January 1 990) 



Manufacturing (Heavy) 


60.2% 


Administrative Services 


8.7% 


Consulting (Engineering) 


3.9% 


Communication 


6.8% 


Marketing 


6.8% 


Others (Less than 2% each) 


12.6% 
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qorptorate Size At Interview Location 




(N=103 Survey Period May 1989 January 1990) 



500 plus 



100-499 



20-99 



1-19 




33% 



0% 5% 10%15%20%25%30%35% 



B/l Interview Survey 
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New Technologies: Response to ^ 
< open-ended questions 



Combined Use/Need of 15% or More 
Survey Period May 1989 - Jan 1990 



Technology 



Currently 
In Use 



Planned to Use 
(Next 3-5 years) 



Production 

CAD/CAE 40% 
CAM 22% I 

NC 

CNC 36% 
Robotics/Machine Centers 28% 
Jigs & Fixtures 

Flexible Manufacturing (Cell/System) 
Laser Cutting & Control 
Alternative Materials 

Quality 

Lasers (Measurement) 
Production/Product Testing 
MRP II 

Electronics 
Material Control 



31% 



23% 



9% 




' '■ KV>>>V'/>:<.VV > '.WAV 



9 

21% 



16% 



18 %l 
15% 



5% 



18% 



7% 



]12% 



16°/e 



HBl 8% 




118% 



JIT 

Optical Scanning/Coding 
Distribution 

Computerized Inventory 



16°/c 



27°/c 




11% 
13% 



11% 
110% 
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New Technologies; Response to 
open-ended questions 



Combined Use/Need of 15% or More 
Survey Period May 1989 - Jan 199U 



Ted 



Currently 
In Use 



Planned to Use 
(Next 3-5 years) 



Business 

Word Processing 
Information Storage/Retrieval 

Communications 

Electronic Mail/FAX 
Telecommunication 
Data Link (Networking) 

Computer Microprocessor 

Applications 35 <^ 
Scheduling (Personnel/Product) 
Personal Computers 
Billing 
Accounting 
On-line Applications 

Advance Technology 



15% 
15% 



■6% 
i 8% 




21% 
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New Skills & Competencies Employees 
Need: Response to open-ended questions 



N=103 Survey Period May 1989- January 1990 



Additional Skills/Compefencic 
Computer Applications 



38°/c 



40°/c 



Employees 



Current 



General 

Published/Special 

Designed Software 
Manage Array of Activities 
Control Machine Software 
Programming 
Data Management 

Computer 
Operator Skills 

Kevboardinq 

Math 

Applied 
Basic 
Advanced 

Thinking Skills/Problem Solving 

Critical Thinking/Reasoning 1 6% 

Troubleshooting/Problem Solving 27% 



7°/c 



New Entry Level 



ass^ 2 7*%> 




16% 



21% 
16% 




^^^^s^^^^! 2 7 % 



25% 



43% 
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New Skills & Competencies Employees Need: 
Response to open-ended questions 

.". ' - • • ■ • . • • t 



N=103 Survey Period May 1989-January 1990 



Additional Skills/Competenc'io* 



Employees 



Current 



Communications 

General 
Oral 

Visual/Reading 
Writing 

Te injcaj 

Literacy 

Proficiency 48% 
Applications 

New Manufacturing Processes 
Mechanical Background 
Techniques 
Certification 

Work Ethics 

General 

Attendance/Punctuality 
Cooperation 
Self Management 
Time Management 



8°/c 



New Entry Level 




mmzmm 1 Q% 



i 9% 



0% 
0% 

2% $m 5% 

0% 
0% 



■Mmm&s>mm 14% 
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New Skills & Competencies Employee^ $$e 
. Response to open-ended questior^ 



N=103 Survey Period May 1989- January 1990 



Additional Skills/Competencic 



Emp 

Current 



oyees 

New Entry Level 



D evelopmental 

Attitudes 
Common Sense 
Creativity 
Comprehension 
Dexterity 

Follow Instructions 
Inquisitiveness 

Interviewing 
Know How to Learn 
Physical Strength 
Resume Writing 
Spatial 

Self Confidence 
Willingness to Learn 
Adaptation to Change 
Self Motivation 



Electronics 



42% 



1 5%t 




a 6% 



W 13% 



o%m 3% 

°°/°Sl4% 
0%— 10% 

o%bn 6% 



10% 



o% 



o% 



2% 



114% 



|2% 
0%|1% 

0% 
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U 12% 
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Anticipated Sources New Technology 
Related/Skills Training Anticipated 



N=103 Survey Period May 1989 - January 1990 



Technical College 
In-plant (informal) 
In-plant (formal) 
Vendor/Manufacturer 
University (Scheduled Courses) 

Secondary School 
Prior Experience 
Tuition Reimbursement 

Seminars 

Workshops, Shows, & Conferences 

Manuals 
Corporate Out of Plant 
Consultants 
Apprenticeship 
Professional Organizations 
Customized Training 




%70% 



1 73 
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Current or willingness to work with local schools. 



Tours for Students/Teachers 
Serve on Advisory Committee 

Provide Visiting Speakers 
Provide a Student Work Experience 
Provide a Co-op Education Work Site 
Provide a Shadowing Experience 
Provide a Teacher Work Experience 

Teach a Course 
Others: Articulation & Provide Resources 




0% 10% 20% 30% 40% 50% 60% 70% 80% 



Willingness to provide intern experience for teachers. 
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HIGH TECHNOLOGY TRAINING 
MODEL MODULE 



Planning and Conducting 
Fol low-Up Studies 



Developed as a part of the High -Technology 
Training Model for Rural Based Business and 
Industry, Technical Colleges and Local and State 
Educational Agencies under Grant No. V199A90151. 



Prepared by: 
OrvilJ.e Nelson 
Center for Vocational, Technical and 
Adult Education, University of Wisconsin-Stout 
Menomon.'.e, WI 54751 
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Fol low-Up study 
Introduction 

This module describes the procedures and actions required to 
plan and conduct a follow-up study, m some instances, the 
reader is referred to other modules, such as sampling, that are 
used in planning and conducting the various follow-up surveys. 
The body of this module describes the steps and procedures needed 
to plan and complete a follow-up survey. In some instances, 
additional details have been included in the appendices, where 
this is done, the reader will be referred to the appropriate 
appendix. 

Purpose 

The purpose of this module is to provide the information, 
instructions, and procedures required to design and conduct an 
effective follow-up study. 

Objectives 
The objectives of this module are to: 

1. Identify the situations in which a follow-up study is an 
appropriate technique to acquire data. 

2. Provide information and procedures that will allow the 
reader to plan and conduct an effective follow-up survey. 

3. Provide information and instructions on how to develop a 
valid survey instrument. 

4. Describe how to select a valid sample. 

5. Suggest ways to use the survey results. 
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Value of Follow-Up Studies 
The purpose of a follow-up study is to obtain information 
from graduates, employers and others who have information on your 
programs in order to improve your school's educational programs 
and services. The specific types of information desired will 
dictate the groups to be contacted during the conduct of the 
study and the types of questions to be included in the follow-up 
survey instrument. For example, if you are interested in whether 
your graduates have the competencies needed to make the 
transition from school to the world of work, you would contact 
those who have been out of school for one to two years. In 
contrast, if you are interested in determining how well your 
programs have prepared graduates for career changes, family life, 
and other relationships, you would survey those who have been out 
five to fifteen years. Employers can also provide valuable 
feedback on graduates . 

Planning and Conducting The Fo l low-Up Study 
The starting point in developing a follow-up study is to 
identify the objectives for the study. The best way to do this 
is to determine the types of information needed to make your 
curriculum, instructional, and general program decisions. This 
information can be obtained by asking teachers, administrators, 
school board members, and others concerned with your school 
system what information they need to have. This can be 
accomplished by interviewing samples of these people or by using 
focus-group sessions. The focus-group technique is described in 
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working with smfrU groups module. Another useful technique is to 
record the information needed as you do program and curriculum 
revisions. 

The steps described for the planning and conducting of the 
follow-up study are depicted in Figure l. if you have been 
involved with follow-up studies before, you may wish to look at 
the flow chart and determine those areas in which you need more 
information. The procedures in each step are described in this 
section or in related modules, in some instances, there will 
also be additional information in the appendix. A logical 
initial approach would be to read the flow chart and the 
discussion tjiven for each of the steps in the chart. Then, if 
you need more information, turn to the appendices and/or the 
modules . 

Determ ine Need and Purpose 

The first step in conducting a follow-up study is to 
determine why the study should be done. Identify the types of 
information that are needed to justify and improve your programs 
and services. *lso, review the types of information needed to 
answer questions about your programs and services. This 
information will provide the basis for identifying the need and 
purpose of your follow-up study. 

Sources of information can be your school mission statement, 
teachers, school board members, counselors, parents, and 
concerned citizens, input from these sources can be collected 
informally through conversations with people in these groups and 
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Figure 1 Follow-up Study Flow Diagram 
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listening to the questions they raise in various forums. Your 
school's mission statement should be reviewed to determine what 
objectives are linked to students' transitions into the world of 
work and/or advanced education. More formal processes such as 
interviews or focus groups can be used to obtain information from 
people in these groups. 

The purpose of the survey may be very specific and relate 
only to determining the effectiveness of a specific program or 
may be more general and concerned with how well your school has 
prepared its former students for their next stages in life. 
Identify Objectives of The Study 

After the purpose has been defined, specific objectives need 
to be developed for the follow-up study. Each objective will 
identify a specific end product or outcome of the follow-up 
study. The following is an example of an objective for a 
follow-up study. 

Example: Determine the percentage of graduates that are 

employed in an occupation related to their school 
program . 

The more precisely your objectives are defined, the easier 
it will be to develop an appropriate survey instrument. Good 
objectives also provide the basis for an effective follow-up 
survey. A review of the needs identified in step one can help to 
assure that a complete set of objectives is developed Usually 
there will be at least one objective for each need identified. 
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Review Previous Fol low-Up Studies 

Review any previous follow-up studies conducted in your 
district. These can be useful in identifying important results, 
the techniques used, and the types of survey instruments 
employed. Also, it would be worthwhile to review fo.llow-up 
studies completed by other school systems and agencies. In these 
studies, the results may not be as useful as the techniques and 
instruments used by the researchers. 

In addition to previous follow-up surveys, literature on 
survey research can be helpful. This module gives a practical 
approach to designing and conducting a follow-up study. It also 
contains some sample survey instruments and procedure3. A more 
in-depth and varied perspective on survey research can be 
obtained by reviewing some of the references given at the end of 
this module. 

Select Population and Sampling Plan 

Review the purpose of the survey and identify the most 
appropriate population or populations. A population should be 
selected because it will give valid information in relation to 
the purpose and objectives of the survey. For example, if you 
are interested in obtaining information on the degree to which 
your students have competency in making the transition to the 
next level of education or the world of work, it would be most 
appropriate to identify a population of individuals who have been 
out of school for one, or at the most, two years. If you are 
interested in determining how well your graduates make 
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transitions and changes as their careers and personal lives 
change, you will probably want to select a population for the 
study of students who have been out of school ten or even twenty 
years . 

Procedures for sampling and selecting appropriate sample 
sizes are included in the module on sampling. If you need 
information on sampling, please refer to it. in many instances, 
a sample will give more accurate information and at less cost 
than trying to contact and follow-up on everyone in a group. 
However, you must also consider the public relations value of 
doing a follow-up survey. It may be worthwhile to contact all of 
the students from a particular class. This contact indicates an 
interest in them and their opinions. A sample size chart is 
included in Appendix A. 
Obtain Addresses and Phone Numbers 

The next step of the process is to obtain the addresses of 
the people who will be contacted. If a telephone survey is to be 
conducted, you will need telephone numbers. Usually there will 
be the need for a follow-up of a sample of non-respondents, in 
this activity, you will probably need to call the people selected 
for contact. Therefore, telephone numbers will be important even 
though the majority of your contacts will be done by mail. 

Often it is hard to obtain good addresses for a follow-up 
survey. If you are doing a one-year follow-up survey, you may be 
able to contact family and friends of the former students. Also, 
ask coaches, teachers, and staff for information on addresses. 
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When a five or ten-year follow-up survey is being conducted, it 
is more difficult to obtain valid addresses. Some groups have an 
active reunion committee that keeps an updated list of addresses. 
Check to see if one exists for the class or group that you wish 
to contact. Also, when students are still in school, inform them 
of the follow-up survey and encourage them to keep their school 
informed of their current addresses. 
Develop/Select Survey Instrument 

For some situations, you may be able to find a survey 
instrument which exactly fits the objectives of your study. In 
most instances, you will need to do some modifications to 
existing survey instruments. A word of caution also needs to be 
given here. If you want to compare your results with previous 
follow-up surveys or with those conducted by others, you will 
need to use basically the same questions as used in the previous 
survey. This does not mean that you have to use the entire 
survey instrument; however, the questions on which you want to 
make longitudinal comparisons need to be the same and the 
directions for responding need to be the same. 

It is usually helpful to review survey instruments used in 
other follow-up surveys. This process can suggest ways to 
collect information related to your objectives and provide some 
actual survey questions. Sample follow-up survey instruments and 
a cover letter are included in Appendix B. Also if you need more 
assistance in developing your survey instrument, see the modulo 
on Developing Survey Instruments. 
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Include clear directions on the survey. This first set of 
directions should orient the reader to the entire instrument. 
Each section or set of items will probably n.ied more specific 
directions. Try to kee the items or questions as simple as 
possible and do not use too many different types of items. This 
will simplify your survey and require fewer direction . 

Two basic types of items can be used in the survey 
instrument: (a) open-ended and (b) fixed response. Open-ended 
questions provide an opportunity for the respondents to express 
their ideas and suggestions. Including some open-ended questions 
makes the people contacted feel that you are interested in their 
opinions. The problem with open-ended questions is processing 
the responses. These will need to be analyzed by the researcher. 
A computer program cannot be used unless the responses are coded 
by the researcher. An example of an open ended question is: 
"How could our school's guidance program be improved?" 

Questions with fixed responses can be answered more quickly 
and processed easier. The answers or responses are pre-coded, 
and these numbers can be enter id in a computer for analysis. 
This type of question also places the responsibility for 
selecting an appropriate response on the respondents, some 
openness can be added to this type of item by providing an "Other 

" response choice. An example of a fixed rssponse item is 

given below. 
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"How do you feel about the courses you took in high school?" 

(1) Very Satisfied 

(2) Satisfied 

(3) Unsatisfied 

(4) Very Unsatisfied 

A survey instrument needs to be accompanied by a cover 
letter. The cover letter should be on your organization's or 
school's letterhead and signed by a person the people contacted 
respect. Typical contents of the cover letter are listed in 
Figure 2 . 



Fiaure 2. 




Cover letter content 


Paragraph 


1: 


Purpose and importance of the study. 


Paragraph 


2: 


Convince the reader that his/her responses 
are important . 


Paragraph 


3: 


Note that the responses are confidential and 
re-emphasize the importance of the study. 


Paragraph 


4: 


List a contact person and phone numbers, if 
there are any questions. 


Paragraph 


5: 


Statement of thanks, a closing, and sender's 
name and title. 



If you decide to code or identify the survey respondents, 
there are several options. One choice is to run an extra mailing 
label and place it on the survey instrument. An option that 
gives a sense of more confidentiality is to place a code number 
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on the survey instrument or return envelope* if this is done, 
mention its purpose in the cover letter. 

A final content check should be made before the final draft 
is typed. This needs to be done to determine the content 
validity of your survey. Content validity means that the survey 
items are collecting information that is relevant to the 
objectives of your study. 

Your survey instrument should be duplicated using a high 
quality printing process. Pastel colored papers such as buff, 
light yellow, light green, or light blue provide an attractive 
background for the survey instrument. 

If you want to conduct an employer survey, include an item 
in the survey that asks the respondents to list their employer's 
name and address. 
Assemble and Mail The Survey 

The cover letter, survey instrument, and a return envelope 
will have to be assembled and inserted in the mailing envelope. 
The mailing la L or address will need to be placed on the 
envelope as wexl. if the survey also contains the name of the 
person contacted, care will need to be taken to be sure that the 
name on the survey matches the one on the envelope. 

Depending upon the size of the sample, it may be cost 
effective to look at some alternative postage rates to use in the 
mailing. When a large set of surveys is to be mailed, the bulk 
rate will save considerable postage. However, it doe* require 
that the envelopes be placed in zip code order. Also, you must 
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have a bulk mailing permit. Bulk wail does not receive the same 
priority as first class and therefore, it may take longer to 
deliver. Als^, it vfill not be returned if the address is 
incorrect, unless you include, Return Postage Guaranteed or 
A ddress Corr ect 3 m .P^.gugs^ed. However, you will have to pay the 
first class rate to have these returned with the address 
correction. 

Including a business reply envelope in the. mailing saves 
expense. Only those surveys returned will incur a postage cost. 
However, the unit cost on a business reply is higher than 
regular first class mail. Consult your post office or mail room 
on the various options available and tneir costs. Select the one 
that will be cost effective and also give you the service needed. 

When selecting a mailing date, avoid vacation and busy 
periods. It would be best to mail so that the survey arrives 
during the Tuesday-Thursday time period. This would avoid the 
Monday rush and the Friday exit behaviors. 
Process Return s 

If the survey is coded or contains the name of the person 
contacted, you will want to record the returns as they come in. 
Postmarks provide some information on who has returned the 
survey. Also, monitor the number of surveys received each day. 
When this number drops significantly and reaches one or two per 
day, it is time to contact those who have not responded. Also, 
check the surveys to determine if any coding or editing needs to 
be done before they are submitted for data processing. 
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Con tact Non-Ra spnpriPnt-g 

I find that it is usually worthwhile to send a second 
mailing and to include another copy oZ the survey instrument in 
this mailing. After the second mailing, you will need to 
determine if you have enough information. If the return rate is 
high enough, usually 70 percent or higher, you do not need to 
make additional contacts with your sample. Additional mailings 
could be made. However, these probably will not net a large 
number of responses. At this point, it would be more productive 
to select a random sample of non-respondents and try to contact 
them by phone or obtain information about them through their 
parents or friends, A sample of 15 to 20 would be sufficient. 
If there is a time constraint, or if it will be extremely costly 
to contact these non-responOents by phone, a smaller sample could 
be used. The original survey should provide the base set of 
questions used in the telephone survey. It may be necessary to 
reduce this set of questions in order to make it fit within the 
length of time that is appropriate for a telephone interview 
(10-15 minutes), if that is the case, select the most critical 
questions. Information from the telephone interviews should be 
processed separately. This will allow you to contrast the 
results from the follow-up of non-respondents with the results 
from those who completed their mail surveys, if the results are 
similar, you can generalize your results to the entire population 
surveyed, if they are significantly different, you will need to 
adjust the outcomes in order to make them reflect the 
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characteristics and feelings of the entire population. NOTE: If 
an employer survey is to be conducted, it can be initiated at 
this point. Sample employer surveys are included in Appendix C. 
Analyze Survey Data 

The data analysis used should be selected based on the study 
objectives. For example, if one of your objectives is to 
determine the percentage of former students that is satisfied 
with your school programs, the responses to the question related 
to program satisfaction would need to be summarized and the 
percentage of students selecting satisfied and dissatisfied 
responses would need to be calculated. 

In some instances, your objectives will require comparisons. 
For example, an objective concerned with identifying the income 
differentials between men and women would require a breakout of 
your data by males and females. Usually this is called 
cross-tabulation in statistics. Many statistical programs 
available for analyzing survey data have this capability. Be 
sure to check with the people who will be doing your data 
analysis to be sure that the software used will have the capacity 
to do the analyses you need. Also, ask them to .review the 
printouts with you so you are sure of their contents. 

For some questions it will be appropriate to use statistics 
that indicate the typical response. The mean can be used to 
identify the typical response on a scale that has equal response 
units. For instance, the average income of the respondents 
would be an appropriate application of the mean. Also, if a 
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rating scale format is used to evaluate certain courses or 
services in your school program, the average response would be a 
way of determining the level of satisfaction with each 
characteristic, m Appendix B, statistics appropriate for 
various types of items are listed. 

Another measure of typical value is the median. The median 
is the point at which one-half of the responses are above and 
one-half of the responses are below, it can be used in 
situations where the response intervals are not of equal value. 
For example, the median level of education would be an 
appropriate application of this statistic when the survey 
question asks for the highest degree or level completed. 

The mode is the third statistic used to reflect typical 
value, it is the response choice selected most frequently. The 
mode is useful in situations where the responses to a question do 
not form a continuum or scale. For instance, the mode would be 
the appropriate statistical measure for reporting the city, 
region or area in which most of your former students now reside. 

In some instances it is desirable to report the variability 
in the responses as well. The range of incomes, for example, may 
be as interesting as the average income. The range is probably 
the most commonly used measure of variability. People hear and 
see the ranges in temperatures each day as they view the weather 
news on TV. They also have an opportunity to experience the 
variations in temperature during the day and thus have a sense of 
what this variability means. The range is the difference between 
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the largest and the smallest values. To illustrate this, suppose 
that the incomes reported had a high value of $100,000 per year 
and low value of $8,000 per year. This would have a range in 
incomes of $92,000. It would indicate that there is a wide range 
of purchasing capacity in the sample. 

Another measure of variability is the standard deviation. 
It is an indicator of the degree to which individual values vary 
from the mean of the total set of values. The standard deviation 
has a more complex formula than any of the statistics mentioned 
so far. Typically it would be calculated by computer. A 
standard deviation of zero indicates that there is no 
variability, in other words, that all of the values are the same. 
As values spread out from the mean, the standard deviation will 
become larger. The maximum value a standard deviation can have 
is one-half of the range of t. •=». values. 

This has been a brief introduction to basic descriptive 
statistics. A basic statistics book should be consulted for more 
indepth information on these statistics and to explore a wide 
variety of additional statistics available. A more efficient 
approach may be to consult with a person who specializes in 
statistical analysis. This individual can quickly suggest tue 
appropriate types of statistics and statistical analyses that 
would be appropriate for your objectives. 

One last suggestion. Include statistical analyses that will 
be understood by your audience. It may be necessary for you to 
have more sophisticated analyses run to answer some of your study 
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objectives. However; do not forget to obtain frequency counts, 
percentages, averages, medians and modes from your computer runs. 
These will be more meaningful to many of your audiences. 
Write Report 

Two major factors need to be considered when developing the 
report or reports from a survey. First, what are the 
characteristics and information needs of the audience(s) that 
will receive the report? Second, what are the objectives of the 
study? 

In most situations, there will be several audiences that 
will have an interest in the survey results. Parents, teachers, 
administrators, local businesses and industries, and graduates 
are typical audiences for a follow-up report. There is probably 
a core of information that will be of interest to most, if not 
all, of the audiences. Also, there will be some pieces of 
information that will have special value to specific groups. 
Before writing any reports, it would be a good idea to look at 
each of the potential audiences and identify their primary 
interests and the types of information they can process most 
readily. After this is done, a specific audience can be selected 
an;' the report writing can be initiated. 

Many audiences are concerned about the outcomes of the study 
and are not as interested in the details on how the study was 
conducted. For these audiences, report the major outcomes first. 
Place those items of greatest interest first in the report. 
Write in a conversational style. A paragraph or two at the end 
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of the report can be devoted to some of the major details of how 
the survey was conducted and who was involved in the survey. 

You will probably need to do a more formal report of the 
results also. For one thing, this is valuable for the historical 
records of your school or agency. The formal report should 
contain the following sections: (a) Cover Page, (b) Abstract, 
(c) Table of Contents, (d) Part I - Purpose and Objectives, (e) 
Part II - Review of Previous Research, (f) Part III - Survey 
Methodology, (g) Part IV - Results, (h) Part V - Recommendations, 
(i) References, and (j) Appendices. 
Disseminate Results 

Usually, people who conduct follow-up studies do not 
disseminate the results. Thus, a proactive approach is 
important. Identify the groups and individuals who could profit 
by learning more about the results of your study. Then try to 
identify ways and situations in which the results can be 
presented to them. If possible, make a presentation to the 
group . 

Visual aids that present key results are also helpful. A 
brief handout that contains the major results presented provides 
an effective supplement to the presentation. 

Sample brochures that reports the results of the follow-up 
study and a composite report for all high schools participating 
in the High-Technology training Model Project are included in 
Appendix D. The brochures present the major findings in a 
concise and attractive format. A copy of the brochure can be 
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included in mailings to parents, taxpayers, and other interested 
groups . 

Do not overlook distributing your results to local media. 
Often local newspapers, radio stations and television are 
interested in information such as you have in your follow-up 
report. Contact the paper or station and talk to the reporter 
who will present the results. 

Be sure to share the results with teachers, counselors and 
administrators. Also, some of the results will be relevant to 
current students. 

These have been some suggestions and ways to disseminate the 
results. Use your creativity to identify dissemination 
strategies that will effectively reach people who need to know 
about your programs. Do not be afraid to try something 
different. 
Implement Changes 

One of the most effective ways of implementing change is to 
involve the people who will need to change and make changes in 
the process of identifying what modifications need to be made in 
existing courses, programs and services. 

Careful attention should be given to the factors and 
conditions that motivate change. Behaviors that are critical to 
the change process need to be reinforced. For instance, interest 
in changing technology could be reinforced with travel funds to 
visit area industries that are implementing new technology, 
attend conferences that present new technology, and/or visit 
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school programs that have developed new curriculums. 

After your staff has had an opportunity to review the 
follow-up study results and learn about new programs, they should 
be involved in discussion groups to identify needs and 
priorities. Once this has been completed, they can help 
establish objectives and select strategies to accomplish these 
objectives. 

Resources are also an important part of change. Many good 
ideas for new programs have been stymied by the lack ot resources 
to implement them. Additional funding is helpful; however, 
expanding access to community resources can facilitate change. A 
variety of businesses, industries, and individual talents can be 
tapped with little or no cost. Talk with local business people 
to determine how they will help your school. 

Often the plan for change gets hung up with the barriers. 
It is important to identify the objectives, where you need to be, 
for your programs. After this has been done, brainstorm ways to 
achieve those goals. Do not let the barriers or problems 
interfere with the brainstorming process. They can always be 
factored into your planning later. 
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Sample S:' . Chart 
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APPENDIX B 
Follow-Up Survey Instrument for 
Former Students 



*Sample Guidelines 
*Sample Cover Letter 
*Sample One-Year Follow-Up Survey (Form BG-1) 
*Sample Five-Year Follow-Up Survey (Form BG-5) 
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Appendix B 



Sample Guidelines for Developing a 
Fol low-Up Survey Instrument 



A. Types of Information Collected 

- Employment history 

- Value and relationship of high school preparation to 
current job 

- Satisfaction with high school courses 

- Past high school educational experiences 

- Adequacy of high school preparation for life skills and 
situations 

- Adequacy of career planning assistance 

B. Appropriate Statistics 

(Items from the one-year follow-up survey are used as 
examples . ) 



ITEMS 



STATISTICS 



5-7, 



13 



frequency (f) and percent 



14, 16, & 17 



8, 12, 15 



mean or median 



19-44 



46 



Categorize and tally the 



responses. Report the 



category and frequency the 



response is given. 
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SAMPLE COVER LETTER 
(Place this on your letterhead) 

April 3, 1989 



Dear Graduate: 

Our high school is conducting a follow-up study of graduates in 
order to determine how effective our courses and services are. 
As a former student, you can provide valuable information on how 
well we prepared you for your present activities and 
responsibilities. The enclosed survey will take about ten 
minutes to complete. On most of the items you can check your 
responses. There is space at the end of the survey for you to 
make comments. We would like to have you include any thoughts or 
suggestions that you want to add. 

All of the information on this survey will be process by 
computer. Only summaries of the survey responses will be shared 
with our teachers, administrators, counselors and school board. 
Your responses will be kept confidential. 

If you have any questions, please contact me. 

Sincerely yours, 



Signature 
(Staff Member) 

(If possible, have a staff member the graduates know sign the 
letter) 

Enc. 
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CVTAE/ H IQH TECH 
(III 7294.U1-1 - Mod) 



ONE-Y^AR FOLt-OW-UI' SU MVEY, 



Fqrni PQ -1 



Pago t 



Wo aro presently doing an o valuation ol our school district's programs. Wo want to improvo thoso programs and also wish lo 
dotermino job and educational prcparodnoss ol all graduates. Your honost input will bo appreciated. 



Studont Name 



High School 



5. (1). 



Malo 



(2) Femalo 



ti. I an prusontly: (Phase chock qqq o/ thoso following 
sovon statements.) 
Full time omployod: 32 hrs/weok or more; 
paiMimo cmployod: less than 32 hrs/week. 

(1) employed FULL time in a nonmilitary occupa- 
tion. 

(2) omployod FULL time in the military sorvice. 

(3) employed PART time--no1 in school. 

(4) employed PART time while in school. 

(5) lull-time siudoni-not employed. (If checked 

go to #15.) 

(6) unemployed but looking lor paid employment. 

(It checked go to tn 5.) 

(7) , unemployed, not looking lor paid employment. 

(If checked go to #15.) 

7. Do you own or operate? your own business? 
(1) t Yes, full-time 



(2) 
(3) 



, Yes, part-time 
' No 



8. What is your present hourly wage including tips? (Do 
not include overtime) 

$ /hour # hours/week 

If you are nol paid on an hourly basis, please rospond 
as follows: 



$ amount 

9. (1) Job Title 

(2) Job Duties 



number ol hours per ( ay period 



(3) Zip code of city in which you work , 



10. May we contact your supervisor in our Employer 

Follow-up? (His/her response wilt be kept anonymous) 

(1) Yes „ 

(2) No 



(your signature) 



11. Supervisor's name: 

Business n^me: 

Address: 



12. Did your high school preparation help you to obtain your 
present job? (Check one) 



O) 
(2) 
(3) 
(«). 



Considerably 
Somewhat 
Very little 
None 



13. Is this job related lo your area ot high school 
preparation? (Check one) 

(1) directly related 

(in s.ime job area as high school preparation) 

(2) somewhat related 

(using some skills learned in high school 
courses) 

(3) QUI related 



(for School Use) 
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14. What parts ol your high school preparation have been ol 
greatest value in your prcsont job? (Check all that apply) 



0). 

(2) . 

(3) . 
<«). 
(5). 

(7) . 

(8) . 
(9) 



. General job skills (work habits, wotking with 
others, etc.) 
. Spocilic job skills 
. Career awareness and information 
. General preparation 
. English skills 
. Math skills 
. Science skills 
. Computer skills 

Othor _ 



15. How do you leel about the courses you took at your high 
school? (Check one) 

( 1 ) I was very Satislied 

^2) I was Satisfied 

(3) I was UNsatii>iied 

(4) I was very UNsatislied 

16. Do you leel vocational courses including career explora- 
tion and the development ol saleable job skills should 
be part ol the high school program? 



0). 
(2). 



.Yes 
Not sure 



(2). 
(3) 



(3) No 

17. Are you currently attending a school/educational 
program? (Check one) 

0) a GEO (High School Equivalency) program 

. a vocational-technical school diploma 

program one-year or loss 
m a vocational-technical school two-year 
associate degree program 
. a two-year UW-Center program 
. a four-year university or college program 
. an apprenticeship program 
„ an in-service or on-the-job training program 
. private school program 
. I am not attending school 
other 



(4) . 

(5) . 

(6) . 

(7) , 
(6). 
(9). 

(10). 



18. II you checked a school/educational program in #1 7, 
please list your current program/area of study. 



Name of current program/area of study 



Name of school location 
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In general, how adequate was your high school preparation in tho following situations or Hfo skills: (Pioaso circle number 5, 4, 3. 
2 t Q[ 1 (or each statement.) 



More Than 
Adequate 



Adoquato 



Not 
Adequate 



19. Seeking a job a 

20. Loaming spocilic job skills S 

2 1 . Learning specific job knowledge 5 

22. Getting along with others 5 

23. Doing able to talk to the boss about your problems 5 

24. Understanding the value and importance ol work 5 

25. Solving problems & 

26. Understanding the U.S. and wo;ld economic systems (supply -demand, 
inflation, rccossion. etc.) 5 

27. Understanding how a business ma^os a profit 5 

28. Understanding alternatives tor solving homo and lamily rotated problems 5 

29. Understanding child growth and development and parenting responsibilities. . 5 

30. Managing caroer responsibilities with home and family responsibilities 5 

31. Applying basic skill preparation in main, reading, writing, spelling, speaking, 
listening to every day problems. - 5 



3 
3 
3 
3 

3 
3 
3 

3 
3 
3 

3 
3 



2 
2 
2 
2 

2 
2 
2 

2 
2 
2 

2 
2 



1 
1 
1 
1 

1 
1 
1 

1 
1 
\ 

1 
1 

1 



In general, how adoquato was tho career planning assistance and information you received at your high school? (Pioaso circle 
number 5. 4, 3. 2 % Qf I tor each statement). 



Moro Than 
Adequate 



Adequate 



Not 
Adequate 



32. Determining my skills, interests and duiiiiios 5 

33. Opportunity to hear speakers Irom the world ol work. 5 

34. Presentation ol role models in nontraditionai area? {i.e., female 
mechanics/male nurses) 5 

35. New and/or future oriented occupations & 

36. Jobs held by primarily males or primarily females 5 

37. Occupations which have local job opportunities 5 

30. Jobs which become outdated, overcrowded or ulfer no opportunities for 

advancement 5 

39. Post-high school educational opportunities related to my career goal 5 

40. Job alternatives related to my career goal 5 

4 \ . How to find a job/how to apply lor a job 5 

42. How to start a business 5 

43. Selecting a career * 5 

44. Adapting to change 5 

* 5. What is your homo zip code? , , 



3 
3 

3 
3 
3 
3 

3 
3 
3 
3 
3 
3 
3 



2 
2 

2 
2 
2 
2 

2 
2 
2 
2 
2 
2 
2 




47. Yos 



No 



Would you like a copy of the 



survey summary? It yes, please give your name and 
mailing address. 



Name 



City 



State 



Zip 



Thank you (or responding. Please return this completed survey in tho enclosed prepared envelope. Add any additional 
comments on tho back oi tho cover loUor and also return. 
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Wo aro presently doing an evaluation ol our school district's programs. Wo want to improve thr.so programs and also wish to do- 
tormino job and educational preparedness ol all graduatos. Your honost input will bo appreciated. 



Student Name 



High School 



SIAQIJiSae 
5. (I) _Mala 



(2). 



female 



6. I am presently: (Please check qqh of theso following 
seven statements.) 

Full-lime employed: 32 hrs/weok or more; 
pan-time employed; less than 32 hrs/weok. 

(1) employed PULL time in a nonmilitary 

occupation. 

(2) employed FULL time in the military service. 

(3) employed PART time -not in school. 

(4) employed PART time while in school 

(5) full time student-not employed. (If chocked 

go tofftS.) 

(8) unemployed but looking lor paid employ 

moot. (If checked go to #15.) 

(7) unemployed, not looking for paid employ- 
ment. (If checked go lo Pi 5.) 

7. Do you own or operate your own business? 



0). 

(2) 
(3). 



Yes, full'time 

Yes, part-time/second income 
No 



8. What is your present hourly wage including tips? (Da not 
includo overtime) 

$ /hour # hours/week 

II you are not paid on an hourly basis, please respond as 
follows: 



number of hours per pay period 



$ amount 

9. (1) Job Title _ 
(2) Job Duties 



(3) Zip code of the city in which you work 



10. May we contact your supervisor in our Employer Follow- 
up? (His/her response will be kept anonymous) 



0). 
(2). 



Yos 
No 



(your signature) 



11. Supervisor's name: 

Business name: 

Address: . 



t2. Is this job related to your area of high school 
preparation? (Check one) 

(1) directly related 

(in same job area as high school preparation 
training) 

(2) somowhal re'atod 

(using some shlls learned in high school 
courses) 

(3) qqi related 



(For School Use) 



4. 



IPC 
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4 


WNM 

5 



14. What parts of your high school preparation have boon of 
groatout valuo in your present job? (Check all that apply) 

0) General job skills (work habits, working with 

others, etc) 

(2) Specific job skills 

(3) Career awareness and inlormation 

(4) General preparation 

(5) English skills 

(6) Math skills 

(7) Science skills 

(8) Computer skills 

(9) Other 

15. How do you loot about the courses you took at your high 
school? (Check one) 



(t) 
(2) 
(3) 
(4) 



I was very Satisfied 
1 was Satisfied 
I was UN satisfied 
I was very UNsatisfied 



16. Do you feel vocational courses including career explora- 
tion and the development of saleable job skills should 
be part of the high school program? 



(M 

(2) . 

(3) , 



Yes 

, Not sure 
No 



17. Since leaving high school, (Check alt that apply) 



0) 
(2) 
(3) 



. I have received one or more job promotions 
. I have changed jobs one or more times 
. t am doing the same work that I did 
immediately following high school graduation 



18. Since leaving high school, I have completed or am 
presently enrolled in: (Check all that apply) 

(1) a GEO (High School Equivalency) program 

(2) a vocational-technical school diploma program 

one-year or less 

(3) a vocational-technical school two-year 

associate degree program 

(4) a two-year UWOntar program 

(5) a four-year university system or college program 

(6) an apprenticeship program 

(7) an in-service 'a on-the-job training program 

(8) private school |>'ogram 

(9) I have not taken any additional training 

(10) other 

19. II you chocked a school/educational program in #16, 
please list your most current program/area of study. 
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13. Did your high school preparation help you to obtain your 
present job? (Check one) 



(2). 
(0- 



Considorably 
Somewhat 
Very Milo 
None 



Nanw of current program/area of study 



Nnme of school ♦ location 
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In general. how adequate was your High school proparalion in the following silicon* or lilo skills: (Pteaso circle number 5, 4, 3. 
2. Qi I (or each statomont.) 



Moro Than 
Adequate 



Adequate 



Not 
Adequate 



20 Seeking a job j| 

2 1 . Learning spoalic job skills J 

22. Learning spc?afic job knowledge & 

23. Goitmg along with others J 

24. Doing ablo lo talk lo Iho boss about your problems 5 

25. Understanding iho value and importance ol work 5 

2G. Solving problems J 

27. Understanding the US and world economic systems (supply demand, 

inflation, recession, etc.) j| 

20. Understanding how a business makes a prolit J 

29. Undemanding allornalives tor solving homo and family related problems 5 

30. Understanding child growih and development and parenting responsibilities. . 5 

31. Managing caroer responsibilities wilh home and lamily responsibilities 5 

32. Applying basic skill oreparalion in math, reading, writing, spelling, speaking, 
listening lo every day problems 5 



4 
4 
4 
4 

4 
4 
4 

4 
4 
4 

4 
4 



3 
3 
3 
3 

3 
3 
3 

3 
3 
3 

3 
3 



2 
2 
2 
2 

2 
2 
2 

2 
2 
2 

2 
2 



In general, how adequate was Iho caroer planning assistance and inlormalion you received at your high school? (Please circ le 

number 5, 4. 3 t 2 t Q[ t (or each statement). — — 

Moro Than Not 

Adequate Adequate Adequate 



33. Determining my skills, interests and abilities 3 

34. Opportunity k hoar speakers from the world ol work 5 

35. Presentation of role models in nontradilional areas (i.e., female 
mechanics/male nurses) 5 

36. New and/or future oriented occupations 5 

37. Jobs held by primarily males or primarily females 5 

38. Occupations which have local job opportunities 5 

3V. Jobs which become outdated, overcrowded or offer no opportunities for 

advancement ^ 

40. Post-high school educational opportunities related to my career goal 5 

41. Job alternatives related to my career goal ■ -"■ * 5 

4<2. How to find a job/how lo apply lor a job 5 

43. How to start a business 5 

44. Selecting a careor 5 

45. Adaptag lo change* 5 



4 
4 

4 
4 
4 
4 

4 
4 
4 
4 
4 
4 
4 



3 
3 

3 
3 
3 
3 

3 
3 
3 
3 
3 
3 
3 



2 
2 

2 
2 
2 
2 

2 
2 
2 
2 
2 
2 
2 



4G. What is your home *p code? . 



47. What additional skills and knowledge did you have to learn in order to keep up with changes in your job. 



■18. Additional skills or knowledge you will nood to learn in the near future in order to keep up with changes in your job. 





49. Yes No _ Would you like a copy ol the 


Name 








survey summary? If yes, please give your nano and 










mailing address. 












City 


Stale 


Zio 





1 think you (or responding. Please return this completed survey m the enclosed prepared envelope. Add any additional 
comments on the back ol the cover letter and also tetum. 
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APPENDIX C 

Employer Survey Instrument Development 
*Employer One-Year Follow-Up Survey (Form CI) 
*Employer Five-Year Follow-Up Survey (Form C2) 
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Appendix C 



Employer Survey Instrument Development 

(Based on the Wisconsin Secondary Vocational Program 
Evaluation Employer Form.) 

Types of Information Collected 

* Occupational Competencies 

* Basic Skills 

* General Occupational Competencies 

* Linkages With Schools 

* New Technologies Coming Into the 
Workplace 
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Oear Supervisor! IUnplo\er . 




to contact 
preparedness of 
' in our 



Employee 



High School 



Area of High School Vocational Preparation 



(For School Use) ' 



IPC 



sequi-no: 



GE 



AC. 


HI: 


IP 


HI: 


Ml-: 


if- 


1 


2 


3 


4 


5 


6 



3 | HlDLllPl 

4. 



Malcl Female 
Graduate 



5. 


rr| pt 




Graduate 



6. 



I 2 I 3 I 4 I 5 | 



Please rate the individual in the following areas: (Circle the appropriate responses) 

START HKRF f 



Well Adequately Noi at all 
Prepared Prepared Prepared 
(5) (4) (3) (2) (1) 



7. Occupational knowledge 

8. Occupational skills 

9. Work attitude 

10, Work quality 

1 1. Work quantity (amount produced) 



5 
5 
5 
5 
5 



4 
4 
4 
4 
4 



3 
3 
3 
3 
3 



I 



In the following areas of basic skills, how well prepared was this employee? (Circle the appropriate 
responses) 



Well Adequately Not at all Not 
Prepared Prepared Prepared Observed 
(5) (4) (3) (2) (1) (0) 



12 Reading 

13. Spelling 

1 4. Computer literacy 

15. Math 

16. Science 

17. Writing 

IK. Speaking 

19. Listening 

20 Problem solving . 



5 


4 3 


2 1 


0 


5 


4 3 


2 1 


0 


5 


4 3 


2 1 


0 


5 


4 3 


2 1 


0 


5 


4 3 


2 1 


0 


5 


4 3 
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Please rate this employee on each of the characteristics listed below. (Circle the appropriate 

responses) Adcqua tcly Not at all 

Prepared Prepared Prepared 
(5) (4) (3) (2) (I) 



Not 
Observed 
(0) 



21. Use of tools and equipment 

22. Selection and care of space, materials, and supplies. 

23. Ability to work with co-workers 

24. Accepting advice and supervision 

25. Working in a leadership role when applicable 

26. Understanding the value and importance of work. . 

27. Prompt ana dependable in attendance 

28. Willing to accept and perform tasks 

29. Ability to plan and direct own work 

30. Ability to make sound decisions 

31. Shows consideration and kindness towards others. , 

32. Neat, clean, and appropriate appearance 

33. Flexibility to accept change 

34. Ability to learn new job tasks 



35. What is your overall rating of this individual's occupational 
competence as it relates to his/her job? 



(5). 
(4). 



.Very Good 
Good 



(3) Acceptable 

(2) Poor 

(1) Very Poor 

36. How would you rate this employee's preparation in relation to 
other employees in his or her work group? 

(3) Individual is better prepared 

(2) Both are about the same 

(I) Individual is less prepared 



(0) No basis for comparison 

37. What has been this employee's attitude toward on-the-job 
training? 

(5) Veiy positive, eager to Icam 

(4) Positive, willing to leant 

(3) Acceptant, but needs prodding 

(2) Negative, resists training 

(1) No training has been offered 

38. If the opportunity should arise, would you promote this 
employee? 

(1) Yes 

(2) Not sure 

(3) No 



5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 



4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



39. When hiring recent high school graduates, would you prefer to 
hire persons who have had high school vocational training? 

(3) A good deal of preference 

(2) Some preference 

(1) No preference 



40. Do you feel vocational courses including career exploration and 
the development of saleable job skills should be part of the high 
school program? 



(1) . 

(2) . 

(3) . 



.Yes 
Not sure 
No 



41 . In which of the following ways does your business work with 
secondary schools to support education? (Please check all that 
apply) 

(1) Suggesting new vocational courses 

(2) Recommending equipment and materials to be 

used 

(3) Providing equipment/materials/donations 

(4) Providing facilities 

(5) ^Providing vocational students with work experi- 

ence 

(6) Serving on advisory committees 

(7) Providing teachers with work experience 

(8) Releasing employees to teach vocational courses 

(9) Partnerships 



Please comment on: 

(I) Additional skills you feci are needed by graduates 



(2) How high school vocational programs could be improved 



(3) What new technologies arc coming into your work place 
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III 729H C2 I 
Dear Supervisor/ Employer, 

We ire in the process of evaluating our school districts vocational progremxs and hive received written waiver from your employee to contact you. 
Since this will provide valualHe data for our district with respect to program improvement as well as job and educatiotial preparedness of 
graduates your honest input is vital. Your cooperation will be appreciated. No student, employer or identifiers will be presented in our reports. 
Start with item 7. 



Employee 



High School 



Area of High School Vocational Preparation 



(For School Use) 



IPC 



SEQUENCE 



GE 



2. 



AO 


BE 


HO 


HE 


ME 


TE 


1 


2 


J_ 


4 


5 


6 



3 | HlDiJrol 



4. 
6. 



Maid Female 



Graduate 



FT PT 



Graduate 



ma nil rusn ran esse 



I 1 I 2 I 3 I 4 I 5 I 



Please rate the individual in the following areas: (Circle the appropriate responses) 
START HERE 



Well Adequately Not at all 
Prepared Prepared Prepared 
(5) (4) (3) (2) (1) 



7. Occupational knowledge 

8. Occupational skills 

9. Work attitude 

10. Work quality 

1 1 . Work quantity (amount produced) 



5 
5 
5 
5 
5 



4 
4 
4 
4 
4 



3 
3 
3 
3 
3 



2 
2 
2 
2 
2 



In the following areas of basic skills, how well prepared was this employee? (Circle the appropriate 
responses) 



Well Adequately Not at all Not 
Prepared Prepared Prepared Observed 
(5) (4) (3) (2) (1) (0) 



12. Reading 

13. Spelling 

14. Computer literacy 

15. Math 

16. Science 

17. Writing 

18. Speaking 

19. Listening 

20. Problem solving . 



5 

5 

5 
5 

5 
5 
5 
5 
5 



4 
4 
4 

4 
4 
4 
4 
4 
4 



3 

3 
3 
3 
3 
3 
3 



2 
2 
2 
i 



1 



0 
0 
0 
0 
0 
0 
0 
0 
0 
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Please rate this employee on each of the characteristics listed below. (Circle the appropriate 
responses) 



Well Adequately Not at all Not 
Prepared Prepared Prepared Observed 
(5) (4) (3) (2) (I) (0) 



21. Use of tools and equipment 

22. Selection and care of space, materials, and supplies. 

23. Ability to work with co-workers 

24. Accepting advice and supervision 

25. Working in a leadership role when applicable 

26. Understanding the value and importance of work. . . 

27. Prompt and dependable in attendance 

28. Willing to accept and perform tasks 

29. Ability to plan and direct own work 

3C. Ability to make sound decisions 

3 1 . Shows consideration and kindness towards others. . 

32. Neat, clean, and appropriate appearance 

33. Flexibility to accept change 

34. Ability to learn new job tasks , 



5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 



4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 



3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 



2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 



S5. Whaf is your overall rating of this individual's occupational 
competence as it relates to his/her job? 

(5). _ Very Good 

(4) I Good 

f3>) Acceptable 

(2) Poor 

(1) Very Poor 

36. If the opportunity should arise, would you promote this 
employee? 

(1) Y« 

(2) Not Mine 

(3) No 

37. When hiring recent high school graduates, would you prefer to 
hire persons who have had high school vocational training? 

(3) A good deal of preference 

(2) Some preference 

(1) No preference 



0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 



38. In which of the following ways docs your business work with 
secondary schools to support education? (Please check all that 
apply) 



(1) . 

(2) . 

(3) . 

(4) . 

(5) . 



(6)_ 
(7>- 

(9)_ 



.Suggesting new vocational courses 
. Recommending equipment and materials to be 
used 

.PioWding eqiupmcnt/matcriaJs/donations 
m Providing facilities 

.Providing vocational students with work experi- 
ence 

. Serving on advisory committees 

. Providing teachers with work experience 

. Releasing employees to teach vocational courses 

.Partnerships 



Please comment or: 

(1) Additional skills you feci arc needed by graduates 



(2) How high school vocational programs could be improved 



(3) What new technologies are coming into your work place 
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APPENDIX D 
Reporting of Results 
- Wausau West Five-Year Follow-Up Survey 
School wide Composite Five- Year Follow-up Survey 
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Project Overview 

The High-Technology Training Model Project 
is funded as a Cooperative Demonstration 
Program by the Department of Education. The 
project will develop a model for effectively 
identifying high-tech training needs of 
business and industry, and isolate 
competencies that are relevant to secondary 
and postsecondary cuniculums to correspond 
with the appropriate needs of business and 
industry. 

The enclosed comments are based on the 
results of the follow-up study you responded 
to for your high school. 

If you have any questions please contact Tim 
Mero, Research Associate, High Technology 
Project (715) 232-3793, or the Local 
Vocational Education Coordinator (LVEC) at 
your high school 



The following are the most significant skills 
or knowledge the respondents have had to 
learn in order to keep pace with changes in 
currant job requirements (percent of 
respondents is indicated). 
#47: 18% - interpersonal skills 
14% - communications 
14% - computer skills 
14% - business management 

The following are skills or knowledge the 
largest percentage of respondents indicated 
they must learn in the near future to keep 
up with anticipated change* 
#48: 35% - advanced computer 
30% - basic computer 
26% - interpersonal relations 




Wausau West 
Five Year 
Follow-Up Study 



Results 



This Survey was conducted during the 
spring/summer of 1989 by the center 
for Vocational, Technical and Adult 
Education as pan of the Cooperative 
Demonstration Research to implement 
a High-Technology Training Model 
for Rural Based Business and 
Industry, Technical Colleges, and 
Local and State Educational Agencies 
under Grant No. V199A90151. 



Center for Vocational, Technical & 
Adult Education 

University of Wisconsin-Stout 

Menomonie, WI 54751 
s ^ 
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Wausau West 
5 Year Follow-Up 



Male 49% 
Female 51% 
Total participants 47 



ro 
O 



Present Employment 




Unomployoii 
not looking for p*i4 mflcymmt 

I'ntmplojroi, tootini for p*Jd «mpicym«* 
FuU4im« flu employ** 
Puvtime, «hiU ifl ichool 
Puvrime, ou ki ichool 
Pull-time, miiiury 
FuU'tim*, oonrriliury 

0* .0* 20* SO* 40* 30* 60% 70% 

77% are working full-time and 14% are still in school. 



«• Average hourly wage. 

iFemale 

Avcnp houdy «i|t »f|mNN 

[Total 

I ■ , 

SO $2 14 16 SI 

Total average wage is $7.22, 

a* The parts of high school preparation that 
have been of the greatest value in their job. 



«• Asked if vocational courses including career 
exploration and the development of saleable job 
skills should be pan of the high school program. 




0* 20* ** 40% 10* 100* 

89% of the students felt vocational courses 
should be part of curriculum. 



Since leaving high school: 



Computer iklllj 
SdoiceiiUj 
MiUiakilli 
fr|liih i hill 
Otncrtl prtpintian 
Cuut iwutnai A Info 
SpocUUjobikilli 
Gcnertljobikilli 




doing tune *otk 1 4i4 
foUoviniHS 



chin | ad job* on* or more tinvct 



I or mor« promotfen* 




66% have changed jobs one or more times. 



Female 



1 ■ 1 ' 
30* 



40* 



Male 



FT 1 1 | 



0* 10* 20* 30* 40* 30* 60* 70* 

General job skills, at 60%, were rated of greatest value. 



*? Education and training since leaving high school. 

ao iddltiorttl trtinini 



, Female 
1 J! Male 



«• How do they feel about the courses taken at high school? 

Very UftJitiifttd 




prfviu ichool 
M<vhi jo6»inlnin| 
ippftnceeehip 
4 jut univenity propem 
2 yen UW'CtflUtf 
Vocitiaul, uuocuie depot 
VociUmii, 1 ftu 
CED 



0* 10* 20* 30* 40* 30* 60* 70* 

94% were satisfied with high school courses. 



i r r i f i f i 1 1 1 1 1 r 1 1 > r i | i m i ( f M i ) 
0* 10* 20* 30* 40* 30* 60* 

87% have completed or are actively enrolled 
in a degree program. 
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5 Year Follow-Up Study 
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Present Employment, 



o 



Unemployed, 
na looking for ptid employment 

Unemployed, locking for p*!4 eroploymem 
FuU'timt fludcnt, not employed 
Pirt'ti/ne, while iA ichool 
Put- time, not in ichool 
FulMime, miliu/y 
Riil'time, nonrulitiry 



«• The parts of high school preparation that 
have been of the greatest value in their job. 



«■ Asked if vocational courses including career 
exploration and the development of saleable job 
skills should be pan of the high school program. 





0% 20* 40% 60* 10% 100% 

83% of ;he students felt vocational courses 
should be pan of curriculum. 
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doing ume work I did 
following HS 



Since leaving high school: 



Female 

. Male 

'1 1II' H»M f il l I| » It 1 ( 1 1 11 1) 1 111) 

0% 10* 20% 30% 40% 50% 60% 70% 



67% are working full-time and 22% are still in school, 



changed job* one of mow lime* 



I or more promotion* 



I 1 
0% 



' I ■ 
10% 



■ ) • i i i I i I i I I 1 f I I I 1 I I I | 
20% 30% 40% 30% 60% 



General job skills, at 52%, were rated of greatest value. 



How do they feel about the courses taken at high school? 

Very Unftitftfted 
Unutitfod 
SidiAeo* 
Very iitisfled 

■^^^^^H I imi i | 

0% 10% 20% 30% 40% 50% 60% 70% 10% 

92% were satisfied with high school courses, 





68% have changed jobs one or more times. 



«■ Education and training since leaving high school. 

no idditionil t/iining 



priviie Khool 
on-ihe job-iriining Vjm 
ipprenticeehip J^IJ 



4 yuf university progrtm 
2 yut UW' Center 
Vociiicrul, luodiu dejfM 
Vocitiontl, 1 yu/ 
CED 




nrpmyn n | m i | j i hj n n | n p i ■■ |i nn 
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 

87% have completed or are actively enrolled 
in a degree program. 
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HIGH TECHNOLOGY TRAINING 
MODEL MODULE 



Planning and Conducting 
A Delphi Survey 



Developed as part of the High-Technology 
Training Model for Rural Based Business and 
Industry, Technical College and Local and State 
Educational Agencies under Grant No. V ^ A90151. 



Prepared by: 
J. Timothy Mero and Orville Nelson 
Center for Vocational, Technical and 
Adult Education, University of Wisconsin stout 
Menomonie, Wisconsin 54751 
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Introduct ion 

This module describes the procedures and actions required to 
plan and conduct research using the Delphi survey technique. In 
some instances the reader is referred to other modules such as 
Selecting Appropriate Samp l ing Techniques that are used in 
planning and conducting the Delphi survey. The body of the 
module describes the steps and procedures needed to plan and 
complete a Delphi survey. In some instances additional details 
have been included in appendices, where this is done, the reader 
will be referred to the appropriate appendix. 

Purpose 

The purpose of this module is to provide the information, 
instructions, and procedures needed to plan and complete an 
effective Delphi survey. 

Objectives 
The objectives of this module are to: 

1. Identify the situations in which a Delphi study is an 
appropriate technique to acquire data. 

2. Provide information and procedures that will allow the 
reader to plan and conduct an .effective Delphi study. 

3. Provide information and instructions on how to develop a 
valid survey instrument. 

4. Suggest one way of displaying survey results. 

5. Suggest how to use survey results. 

Value a£ h neiphi study 
Often research, and education, evaluation, or curriculum 
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specialists are asked to identify, gather and develop information 
on an issue, trend, problem, or objective which is not completely 
defined. In other instances the issue, trend, problem, or 
objective may be clearly understood, but little or no direction 
is available regarding the recommended action to take or the type 
of data to collect. In one scenario the intent may be to 
identify and prioritize the high technology trends an institution 
must respond to. In a second scenario, the purpose might be to 
identify and prioritize the high-technology competencies needed 
for a particular group or those to be emphasized in a program at 
an institution. Unless prior research has been done and is 
available for review, one would not have information on the types 
of trends. 

One technique which the writer found to be of value in 
identifying the trends under these circumstances is the Delphi 
Survey technique. Application of this technique in the 
identification of trends will assist in the development of a 
future oriented perspective for the evaluation and assessment of 
programs. The writers have found the application of this 
technique to be equally useful in the development of evaluation 
and research instruments intencfed to identify the factors 
encompassed in a problem area, and the types of questions which 
may be used to collect data. 

The Delphi Survey Technique 

The Delphi Survey technique was developed after World War II 
to provide a process for predicting future events. Since ire 
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technique was released to the public, it has been applied in a 
number of ways for a variety of purposes. Typically, it has been 
used in one of two ways. Either it has been used to gain 
consensus on future events or it has been employed to derive 
consensus on present problems or priorities. 

The Delphi Survey technique is designed to collect expert 
thinking and provide a process for achieving group consensus. 
The process was designed to minimize the impact of personalities 
on the thinking and decisions of the surveyed group or jury. The 
respondents or jury members are selected to provide expertise 
related to the purpose and subject of the research scudy. 
usually, the first round of the Delphi Survey technique starts 
with a series of very general questions related to the problem or 
topic being studied. For example, if a study was being made to 
predict the nature of our society in the year 2005, one might 
include a question, "what will transportation be like in the year 
2005?" The respondents, as a panel of experts, would then write 
comments giving their views of the nature of transportation at 
this future point in time, it is important that the initial 
questions are of a general nature to avoid inadvertently 
directing the thinking of the respondent. 

Appendix A shows the four questions initially asked of the 
members of the High-Technology Gaining Model Project's trends 
forecasting committee. The purpose of the High-Technology 
Training Model Project's Delphi study was the identification of 
trends in the areas of technology, work management systems and 
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societal issues that would occur by the year 2000. Some argue 
that the use of these general questions can change the thinking 
of the jury members; however, some stimulus is needed to direct 
them to the area being studied. 

The responses to the first round of the Delphi Survey 
technique are analyzed and synthesized into a series of 
statements related to the problem or topic being studied. 
Responses are usually identified on a numbered scale which form a 
continuation (see Appendix B for a sample). These numbered 
responses are then sent back to the jury members for their 
evaluation. If one was predicting the likelihood of events at 
some point in time, the respondents would indicate the 
probability of occurrence. After the responses in the second 
round have been given, the researcher summarizes the results and 
determines the area of consensus on each item. The mean, mode, 
or median may be used to identify the consensus response 
depending upon the type of survey and the response scale used. 
The area of consensus may be determined by the distribution of 
•esponses. It could be an area of plus or minus one standard 
deviation from the mean, or it could be the interquartile range 
(the area between quartile 1 and quartile 3.) In some research 
studies, the area of consensus is defined as the category within 
which the mean, median or mode falls. 

For the third round the researcher provides the respondents 
(jury members) with a summary of the responses from the previous 
round. Respondents are provided instructions and asked to 
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consider those items on which their responses are not within the 
area of consensus (See Appendix C for sample of the 
instructions) . For each response which is outside the designated 
area of consensus, the respondents were given the choice of 
changing their responses to one which is within the consensus 
area or of writing an argument for retaining the original 
response. It is important to emphasize that each of the 
respondents make their decisions in private and do not know who 
has made the other responses. Therefore, personalities and 
reputations have minimal impact. Responses on the third round 
are then summarized by the researcher. 

If a fourth or subsequent round is used, respondents have a 
choice of modifying their responses which remain out of 
consensus, by moving them within consensus or to write a counter 
argument for their continues position of nonconsensus. a high 
level of consensus is usually achieved by the end of the fourth 
round, in fact, consensus is usually high after the third round. 
Application o f the Delphi Survey Tsnhniqno 

When developing instruments under conditions of uncertainty, 
the researcher may not have the detail and information to 
specifically determine the contents of the instruments m. r 
development. This is analogous to the case where a researcher is 
attempting to predict the future but is not certain of the events 
nor the likelihood of their occurrence, since the Delphi Survey 
technique has been used successfully in predicting future events 
and gaining consensus, the writers thought that it would be 
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worthwhile to apply this technique to identify near term 
technological developments and applications. The results of the 
survey could then be used in the identification and/or evaluation 
of competencies which members of the work force must be empowered 
with in order to contribute socially and economically, adapt to 
an ever changing work environment, and to be able to influence 
their own future. 

High-Technology Training Model Project 

Perhaps it would be easiest to describe the use of the 
Delphi Survey technique as it was applied to forecast trends as 
part of the High-Technology Training Model Project. Thirty 
individuals, each considered an expert in their field, were asked 
to participate as members of the trends forecasting committee, 
also referred to as the jury. The intent was to maintain a 
broad, cross-sectional perspective for committee membership which 
represented diverse backgrounds, experiences, and interests in 
lieu of developing an indepth perspective by assembling committee 
members representing a single occupational field or industry. 
The committee had a national and regional memberships 
representing a broad cross section of professional organizations 
and agencies representing: (1) agriculture and agribusiness, (2) 
biotechnology, (3) computer software and hardware, (4) curriculum 
development, (5) economic development, (6) employment (state, 
national, private), (7) futurists, (8) insurance, (9) marketing 
research, (10) manufacturing, (11) nuclear energy, (12) organized 
labor, (13) proprietary and public postsecondary vocational and 
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technical institutions, (14) state and proprietary industrial 
training agencies, (15) Administrators, of Vocational, Technical 
and Adult Education institutions/districts, and (16) university 
faculty members. Their response co the four questions (See 
Appendix A) asked in the first round of the study produced 254 
comments. Each response was typed on a 3 X 5 card for further 
processing. After the responses had been transferred to cards, 
the researcher sorted the statements into piles of like 
statements. Each pile represented a potential statement or set 
of statements for the instrument to be used in the second round. 

Usually, it is difficult to distinguish more than seven to 
nine categories on one sort. Thus, if a wide variety of comments 
is receive, as was the case in this study, it may be more 
efficient to identify six to nine piles or catec 'as for the 
initial card sort. After this has been done, eacn pile can be 
further sorted into more specific sub-categories. 

After a tentative set of categories or items has been 
identified, the researcher should review the statements in each 
pile. Although the statements may vary somewhat within a given 
pile, they should represent one major thought. In other words, 
only one major thought or idea should be represented by each pile 
of cards. 

Having verified the homogeneity of the statements in each 
pile, the researcher can proceed to write the question or 
statement for the final instrument. Often, a statement can be 
selected from the pile which is representative of the idea 
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explored. This has the advantage that it will be written in 
terms understood by the persons who will respond to the next form 
of the instrument. 

In the case of the High -Technology Training Model Project, 
the researchers painstakenly analyzed each comment, developed 
categories for grouping comments and synthesized them into 61 
trends statements. 

Frequently there will be a pattern or several patterns 
represented by the trend statements developed through this 
process. In the example used in this module, there were clusters 
of trend statements dealing with a wide range of subjects that 
included: artificial intelligence, automated offices, 
biotechnology, composite materials, computer integrated 
manufacturing, communication networking, electronics, 
transportation, world of work, products and services, global 
issues, government, and economics. However, in most instances, 
it will be easier to process, analyze and interpret the responses 
if they are placed in their natural clusters. In order to keep 
the study manageable, the 62 trend statements were split into two 
parts. Part one consisted of 32 statements dealing with 
technological trends. Part two contained the 29 remaining 
statements in eight clusters in 11 clusters dealing with work 
management and societal trends. Part one contained four open 
spaces numbered 33 to 36, for the respondent to write additional 
trend statements if desired. As a result, the 30 remaining 
statement trend statements dealing with work management and 
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societal trends, were consecutively numbered in Part two as 37 
through 65. 

Following the second round of the Delphi study, a consensus 
area was identified for each trend statement. Committee members 
were then asked to review each of their responses which were 
outside the consensus area and to either change their response 
(bring it within the consensus area) , or justify their position 
of non-consensus in writing. 

The second and third rounds of the Delphi Study was limited 
to part one. In rounds four and five, the committee addressed 
the trend statements contained in part two. 
Designing the Report Format 

Appendix D is a summary of the responses to the 32 
technological trend statements which made up part one. The 
report format depicts: 

The numbered trend statements. 

The seven point rating scale for the likelihood of each 
trend. The scale ranges from: 

1 - indicates a o% likelihood of happening. 
7 - indicates a 100% likelihood of happening. 
The box plot which reflects: 

(a) consensus area (locating along scale) . 

( b ) % of responses within consensus area. 

(c) non-consensus response area above and/or 
below consensus area. 

( d ) 3 of responses in non-consensus above and/or 
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below consensus area, 
(e) range (inclusive) of the responses. 

4. Capsulized respondents' comments for justification of 
non-consensus responses . 

(a) The (+) indicates justification for rating 
above consensus area. 

(b) The (-) indicates justification for rating 
below consensus area. 

In addition to the capsulized comments, the non-consensus 
comments were grouped by trend statement, number and reported in 
their entirety, in a separate part of the actual Delphi study 
report . 

Survey Analysis 

In the analysis of the results, a primary concern was to 
establish consensus or non-consensus for the rating of each trend 
statement. Consensus was established with the use of a decision 
matrix (Appendix E) based on the following criteria. 

1. Percent of responses within consensus area. 

2 . Width of consensus area . 

3. Range of the likelihood ratings for each statement. 

It must be understood that consensus simply establishes the 
location of the committee's concurrence along the likelihood 
rating scale for the trend statement. 
Trends Forecasting 

Appendix F contains the High-Technology Training Model 
Project Trends Forecasting Committee's prediction of high- 
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technology related trends based on a consensus of 88% or greater 
on the 7 point likelihood rating scale. 

Three major concepts involved in the design and utilization 
of an instrument are: (l) validity, (2) reliability, and (3) 
utility. 

Validity . 

Validity is concerned with whether data collected are 
related t< the decisions or actions to be taken. In the example 
just cited, two measures of validity were used. First, a content 
validity check was conducted by contrasting the trends identified 
in each cluster with the typical trends identified in current 
literature. This check ind e xed that the instrument was 
complete. Second, a concurrent validity check was made. Each 
respondent had an opportunity to add additional trends in an item 
labeled "Other." 

Reliability . 

Reliability can be assessed in different ways. For this 
study, the consistency of the response patterns was utilized. 
Utility . 

Utility refers to the degree of effort involved in designing 
the instrument, collecting the data and making a useful 
application of the information obtained. Although it does take 
some lead time in order to query the panel or respondents 
concerning their reactions to general questions, the amount of 
time required for each respondent to complete Round 1 is 
relatively small. 
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In the High Technology Training Model Project's Delphi 
Survey, each respondent was asked to write the three or four most 
important changes they felt will occur in each of the four areas 
in the next ten years. When the responses were pooled, a wide 
vaziaty of statements was obtained. With a smaller jury, one 
might have to ask for more detail. If a large jury is used, 
fevar responses can be requested for each question. If clerical 
time is available to pull the statements from the survey forms 
and place one per 3X5 card, the instrument development process 
goes very quickly. After the statements have been recorded on a 
card, the porting process can be done quickly. Since the 
instrument was designed around a specific problem and the 
questions in the first form was built on functional area in 
research, the results were readily useable in designing the 
second survey. 

Summary 

Experiences of the writers indicates that the Delphi Survey 
Technique can be effectively used to identify, gather and develop 
information concerning issues, trends, problems or objectives 
which are undefined or instructutfal . The first round of a Delphi 
Survey technique can be effectively utilized to identify the 
specific dimensions of the concept to be measured. The 
information provides the basis for a series of statements or 
questions which can be put into the survey instrument for use in 
the second and subsequent rounds. In situations where the number 
or diversity of the responses is unmanageable, the survey may be 
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U N I V I . . H S I F Y O I WISCONSIN 

CVTAE ~ ~ 

CliNri MIOM VOCAflONAI. ri-CHNlCAl AND ADULT LDIJCAIION 



STOUT 



715-232-1302 

July 26, 1990 M E N O M O N I li WISCONSIN b A J f, | 



RE: High-Technology Trends Forecasting Committee 
Dear : 

The responses for the first round of Part Two (Work Management and 
Societal Trends) of the delphi survey have been processed. We are 
now ready to do the second round of this process. In this round you 
will need to check your response on each item against the consensus 
response area for the item. To simplify this process, we have 
identified those items on which your response is outside of the 
consensus area by circling the statement number in red. If you have 
any questions on what you are to do, please call us. Your first 
round survey is enclosed. You will use it as you complete the second 
round. We would like you to return part two of the survey, using the 
c enclosed envelope, by August 10, 1990. The directions for the second 
round are given below. 

I nstructions for Round Two of Part Two 

In round two you will need to do the following: 



1. 



Review your survey to identify those items on which you- 
responses are out of the consensus area. These items have 
been identified with a red circle around the item number. 

2. When you find an item number that has been circled, compare 
your response to the consensus area. The consensus area 
has been identified with a green box drawn around the 
response (s) that are included in the consensus area. On an 
item where your response is outside of the consensus area 
you have your choice of two actions: 

a. You may change your response to the consensus area. 
This can be done by drawing an arrow from your response 
to the number in the consensus area. if there is more 
than one response in the consensus area, draw your 
arrow to the response choice you prefer. 

b. If you do not want to change your response, you must 
write a justification for your response. Any comments 
made on your first j.round have been compiled and lined 
out on your original to preclude confusion with round 
two comments. if there is space on your survey by the 

UNIVERSITY OV WISCONSIN STOUT IS AN IIOIJAI. OPPORTUNITY ANll AFFIIIMA TIVT ACTION UNIVI -HSITY 
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item, write your justification there. If you wish to 
use lined out comments from round one as justification, 
simply circle the comment. If you need more space, we' 
have added an additional form. On that form you can 
indicate the item number in the left hand column and 
write your justification in the right hand column. Be 
sure to list the item number, so we can link the 
justification to the item. 

3. After you have completed the steps above, return your 
survey form to us in the enclosed envelope. 

The consensus areas were determined by selecting the response or 
response choices that were selected most frequently by the committee 
In some instances, there were two adjacent response choices that wero 
selected by the same or similar number of committee members. m this 
instance, we included more than one response choice in the area of 
consensus. 

After the responds have been received from this second round, we 
will run a summary of the consensus predictions and send a copy to 
you. in addition, we will summarize the justifications from each of 
the surveys and present those along with the statistical summary. 
These justifications will not be presented by the respondents' name. 
There will D ust be a complete list of the justifications for each 
item, once this second round is done, you will be able to see what 
the consensus prediction is for each work management and societal 
change and you will also have some justifications for responses that 
do not fall in the consensus area. 

The comments given on the first round survey were appreciated. We 
are in the process of summarizing these and will provide a summary of 
them with the overall report that will be sent to you after this 
round. 1 

If you have any questions on how to complete this second round, be 
sure to call us (Orville Nelson (715/232-1362, Tim Mero 
(715/232-3793). If we are not in, leave word with our secretary, and 
we will call back. 

Your input is sincerely appreciated. 
Sincerely yours, 



Orville Nelson T i m Mero 

Co-Director Research Associate 

Center for Vocational, Technical 

and Adult Education 
218 Applied Arts Building 
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7. Definitely will happen (100%) 
6. Very likely wiil happen (8199%) 
S. Likely will happen (61-80%) 
4. Moderate chance will happen (41-60%) 
3. Unlikely will happen (21-40%) 
2. Very unlikely to happen (1-20%) 
I. Will not happen (0%) 



Artificial Intclligcncc/Exncrt Systems 

1 . Expert systems with systematic 
rules for decision making will 
decrease the need for technical 
experts. 
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Respondent's comments for non-consensus: + Reduced role with employee involvcmr -\ 

- Increase before decrease. 

- Yes, but not in 10 years. 



2. User friendly, expert systems 
will be used in most industry 
and business fields by 2000. 



1 2 



34% 



6 7 



66% 



Respondent's comments for non-consensus: - Change most to many . 

- Yes, but not in ten years. 



Automated Office/Datamation 

3, The use of electronic data bases 
with controlled access, will replace 
conventional record systems in 40% 
or more of the businesses and 
industries. 



1 2 



6% 



6 7 



94% 



Respondent's comments for non-consensus: - Cost factor is a problem for small businesses. 



Note: Respondents' comments for non-consensus 

4 indicates comment of respondent-rating item above consensus area 
- indicates comment of respondent rating item below consensus area 
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A. Administrative support staff 
members will utili/.c integrated 
electronic work stations as pan of 
their everyday functions. 



o 
i 

1 2 



Respondent's comments for non-consensus: 



20 



40 



60 



80 
i 



3 



5 



100 



6 7 



12% 88% 

Administrators (now in 40s) won't know how to connect. 



5. Impact printers will reach 
obsolescence by 2(KX). 



1 2 



3 



6 7 



17% 83% 

Respondent's comments for non-consensus: Technically obsolete, but many will still be in use. 



Words-pcr-minutc and kcyboarding 
skills will be abolished as the 
standard used to identify a person's 
clerical proficiency. 



1 2 



17% 60% 



Respondent's comments for non-consensus: + More likely. 

- Never was the standard. 



6 7 



23% 



7. At least 50% of the computers on 
the market will include voice 
actuation as a standard feature. 



1 2 
I — 



17% 



72% 



Respondent's comments for non-consensus: + Closer than we think. 

- Will still be add on. 

- % too high. 



6 7 
—I 



11% 



Biotechnolog y 

8. Genetically engineered products to 
cure AIDS and some cancers will be 
on the market. 



1 2 



17% 



72% 



'o 



6 7 



11% 



Respondent's comments for non-consensus: + Higher likelihood based on scientific progress. 

- Prevention not cure. 
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There will he a 50% increase in (he 
availability of federal support lor 
biotechnology oseareh to develop 
::nres for disabling and life 
threatenim; diseases. 

V.' 



9 P 



1 2 



6% 



.40 



60 



80 



88% 



5 



6% 



100 



6 7 



Respondent's comments for non-consensus: t 50% increase is small $ amount. 



10. Of the technological advances in 1 2 3 4 5 6 7 

science, genetic engineering will be I 1 

the most controversial. | 

100% 

CAD/CAE 

I L Computer assisted design (CAD) 
will be used for more than 90% of 
all drafting and design tasks in 
business and industry. 

Respondent's comments for non-consensus: + Without it, companies cannot subcontract. 



1 2 



6 7 



6% 



88% 6% 



iiositc Materials 

12. Sixty percent or more of automobile 
components will be manufactured of 
composite materials. 

12% 88% 

Respondent's comments for non-consensus: - % is too high. 



1 2 3 4 5 6 7 
I 1 I 



13. By the year 2000, 20 percent or 
more of the structural steel 
requirements will be met with 
composite materials. 



1 2 



34% 



6 7 



66% 



Respondent's comments for non-consensus: - 20% is too low. 

- Depends on one's definition of structural steel. 



9 

ERIC 



243 



211 



Computer InUm ratcd Mamifarttmni; () 

M. Smart machines, with ' 

self-diagnosis;^ and self repairing, I 2 
module replacement capabilities 
will l>e on line. 



20 
i 



40 
i 



3 



60 80 100 
i i i 

5 6 7 

j 



to 



1 1% 



Respondent's comments for non-consensus: h lixtem of use will be limited. 



15. The Automated Factory, with fully 1 2 3 4 5 6 7 

integrated manufacturing, will be a 

proven operational concept. | 

17% 83% 

Respondent's comments for non-consensus: - Proven technology, but not accepted in U.S. 



16. Visual and contact sensory hardware I 
and software will expand robotic 
applications. 



17% 



Respondent's comments for non-consensus: + Already beginning. 

- Yes, but not in 10 years. 



6 7 

- H 

77% 6% 



17. At least one space lab devoted to 
manufacturing in space (i.e., for 
critical tolerances) will be in orbit. 



1 2 



6 7 
H 



34% 



60% 6% 



Respondent's comments for non-consensus: + USSR has it now, U.S. will soon. 

- 20 years yes, 10 years no. 



18. Computer integrated manufacturing 12 3 4 5 6 7 
(CIM) will be used extensively in 

80% or more of the industrialized | ~J j 

nations. 1 ' 

28% 60% 12% 

Respondent's comments for non-consensus: - Not in 10 years for extensive use of CIM. 

Yes, but not extensively. 
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I ( > Automatic storage and automatic 
retrieval (AS/AR) systems tor pans 
and components, with controlled 
access, will be widely used in 
companies. 



o 
i 

1 2 



20 



3 



40 



17% 



60 
i 



80 
i 



83% 



100 
i 

7 



Respondent's comments for non-consensus: - Not in 10 years. 



20. Scaled down computer integrated 
manufacturing (CIM) systems will 
be in use by 10% or more of the 
"Job Shops' 1 . 



Respondent's comments for non-consensus: - Need more education to make these changes. 

- Definition of job shop would prevent this. 



Communication Networking 

21. Corporate headquarters will have \ L 3 4 5 6 7 

read only memory (ROM) access to . 

the information and production data I 

bases of their subsidiaries. 

17% 83% 

Respondent's comments for non-consensus: + Don't see how major corporation can exist without. 

- Large corporation yes, but not for small ones. 



17% 



6 7 



83% 



1 2 



22. Ninety percent of all companies will 
have their communication equipment 

(electronic mail, facsimile machines, i 

computers, telephones, answering ' 
machines, laser scanners, and 
printers) internally networked as a 
single, user friendly system. 

Respondent's comments for non-consensus: 



34% 



6 7 



66% 



- 90% seems too high. 

- Too many small companies. 

- I would agree for large coq)oration. 



23. A telecommunications link capability 1 
will be a standard feature on portable 
lap-top computers. 



6% 



6 7 



88% 6% 



Respondent's comments for non-consensus. ■»• A must; I had it five years ago. 
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?A Twenty to twenty live percent of 
employees will work out of remote 
locations, linked to their company 
or corporation through 
telecommunications. 



o 
i 

I 2 

I — 



20 



40 



60 
i 



3 



80 
i 



1 1% 



'o 



100 

6 7 
— I 



66% 17% 



Respondent's comments for non-consensus: + More common than most recognized. 

+ Very likely. 

- They won't waul il as work is a socializing activity, 

- Employees waul to be around others. 



25. Work related trav? will be 1 2 3 4 5 6 7 
significantly reduced (20-30%) due 

to improved communication J 

network systems. 

23% 77% 

Respondent's comments for non-consensus: - Yes, but not in 10 years. 

- Face-to-face is important. 

Computer Literacy 

26. English will be standardized as the 1 2 3 4 5 6 7 
international programming language 

for voice actuated computers. J 

17% 83% 

Respondent's comments for non-consensus: - Not necessary with in line translators, 

- Nationalistic interests. 



27. Thought actuated computers will 
have progressed from the 
development phase to a broad range 
of field tests. 



Respondent's comments for non-consensus: - Can't sec widespread application by 2000. 

- Yes, but not in 10 years. 

- Don't see breakthrough coming yet. 

- Only in concept base.* 

+ This is still only advance research phase. 

- 2% now, question 50% in 10 years. 



1 2 3 4 5 6 7 

I 1 I 1 

28% 55% 17% 



28. Reprogrammable optical (laser) 12 3 4 5 6 7 

disks will be the most popular 

media for the storage of permanent j 

and semipermanent records. 



23% 77% 

Respondent's comments for non-consensus: - Not in 10 years. 

% % Not enough knowledge to respond. 
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% 29, Pictures (icons), not speech or 
characters, will become the 
computer interface standard. 



o 
i 

1 2 
|_ 



20 



40 



3 



60 
i 



80 
i 



22% 



72% 



Respondent's comments for non -consensus: (One omit) 

• Not enough knowledge to respond. 



100 



6 7 



Electronics 

30. International standards will have 
* been adapted for electronic interface 
technology. 



1 2 
I — 



4 



6 7 



6% 94% 

Respondent's comments for non-consensus: - Fast moving, but I can't give to the group view. 



Equipment and Facilities 

31.' The flexible manufacturing facility 
* will be like a hanger or gymnasium, 

providing optimal diversity, 
( ' efficiency, economy, and 

adaptability for changing, multiple 
long and short term requirements. 



Respondent's comments for non-consensus: 



1 2 



- The open concept is less expensive. 
+ We have had that for a long time. 
+ Highly likely to happen. 



6 7 
1 



72% 17% 



Transportation 

32. High speed ground transportation 1 ~ 3 4 5 6 7 

will begin to replace short distance 
. air routes. j 



17% • 83% 

Respondent's comments for non-consensus: - Not by 2000 unless begin means testing only. 

- They are still talking about them; without federal 
help, unlikely to be built. 
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APPENDIX E 
Decision Matrix 
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Trends l"or<x:.ist. i n<j Survey 
Consensus Doe i s i on M.itr i x 



Consensu:; Area 
"; w i. Uh i. n 1 nc rcnion Us 



Uanqo 

( 1 no 1 u s i. y o ) Con s ens u s 



TOO 



94 



Yes 



Yes 



1 



C 



v. 



Yes 
Yes 



Yei 



Yes 



Yes 

Yes 

Yes 
Yes 
Yes 
Yes 
Yes 
Yes 

Yes 
"Yes 
"Yes 
"Yes 
"Yes 



Yes 



88 



Yes 



Yes 



Yes 
Yes 
"Yes 
"Yes 
"Yes 
[Yes 

Yes 
"Yes 
"Yes 
"Yes 
"Yes 



83 



Yes 



Yes 



Ye; 



Yes 
"Yes 
"Yes 
"Yes 
"Yes 
[Yes 

Yes 
"Yes 
"Yes 

Yes 

Questionable 
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Yes 



7H 



Yes 



Yon 



Yes 

.Questionable 
Questionable 
No 

Yes 

"Yes 

Questionable 
>!o 
No 



72 



Yes 



Yes 



Yes 



I 



Yes 
[Yes 

Questionable 
Questionable 
"No 
[No 

Yes 

Questionable 
"No 
"No 
"No 



66 



Yes 




Yes 



Yes 



Yes 

Questionable 
Questionab 1 e 
"No 
"No 
[No 

Questionable 
"No 
"No 

No 

No 
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Trends Forecasting Committee 

Trends Forecast 
(Part One: Technological Trends) 



The Trends Forecasting Committee predicted the likelihood of 
the following technological trends happening by the year 2000. 
Forecast are based on a consensus of 88% or greater on a 7 point 
likelihood rating scale. 



1. (Biotechnology) It is likely to very likely (61-99%) that 
of the technological advances in science, genetic 
engineering will be the most controversial. 

2. (Automated Office/Datamation) It is likely to very likely 
(61-99%) that the use of electronic data bases with 
controlled access, will replace conventional record 
systems in 40% or more of the businesses and industries. 

3. (Electronics) It is likely to very likely (61-99%) that 
international standards will be adapted for electronic 
interface technology. 

4. (Automated Office/Datamation) It is very likely to 
definite (81-100%) that administrative support staff 
members will utilize integrated electronic work stations 
as part of their everyday functions. 

5. (CAD/CAE) It is likely to very likely (61-99%) that 
computer assisted design (CAD) will be used for more than 
90% of all drafting and design tasks in business and 
industry. 

6. (Communication Networking) It is likely to very likely 
(61-99%) that a telecommunications link capability will be 
a standard feature on portable lap-top computers. 

7. (Composite Materials) It is likely to very likely (61-99%) 
that sixty percent or more of automobile components will 
be manufactured of composite materials. 

8. (CIM) There is a moderate to likely (41-80%) chance that 
smart machines, with self-diagnosing and self-repairing, 
module replacement capabilities will *be on line. 

9. (Biotechnology) There is a moderate (41-60%) chance that 
there will be a 50% increase in the availability of 
federal support for biotechnology research to develop 
cures for disabling and life threatening diseases. 
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High-Technology Training Model Project 
Trends Forecasting Committee 



Trends Forecast 
(Part Two: Work Management and Societal Trends) 

The Trends Forecasting Committee predicted the likelihood of the 
following work management and societal trends happening by the year 
2000, Forecast are based on a consensus of 86% or greater along a 
7 point likelihood rating scale. 

1. The committee had 100% concurrence that: 

a. (Products and Services: Quality) It is likely to very 
likely (60-99%) that the philosophy of quality being 
everyone's responsibility will have a strong influence on 
corporate human resource development and organizational 
development programs. 

b. (World of Work) It is likely to very likely (60-99%) 
that the majority of the labor force will be employed by 
manufacturing and service oriented business with less 
than 500 employees. 

c. (Work Skill Development) It is likely to very likely 
(60-99%) that managers and technics will have to become 
qeneralists vice specialists , knowing a little about a 
lot of areas, to creatively solve problems and be more 
mobile within organizations. 

d. (Work Skill Development) It is likely to very likely 
(60-99%) that postsecondary technical education programs 
will be highly flexible and adjustable to the skill level 
and learning speed of each individuals. 

2. The committee had 93% concurrence that: 

a. (Management) It is very likely to definitely (80-100%) 
that small businesses and corporations will need to be 
flexible and responsive to changing consumer needs and 
demands . 

b. (Work Skill Development) It is very likely (80-99%) that 
employers will provide more flexible in-house training 
and education to upgrade the skills of present and new 
employees . 

c. (Work Skill Development) It is very likely (80-99%) that 
employees will be required to have expanded basic and 
technical job skills. 

d. (Global Issues) It is very likely (80-99%) that there 
will be a 20-30% rise in international trading 
partnerships. 

e. (World of Work) It is very likely (80-99%) that 
technological advances will create a shortage of applied 
technologists with 2 and 4 year degrees. 

i, - < •> 
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(Work Skill Development) It is likely to very likely 
(60-99%) that remediation education will be a major 
function of adult education. 



g. (Work Skill Development) it is likely to very likely 
(60-99%) that educational institutions, and businesses 
and industries will work as partners with students 
participating in cooperative programs. 

h. (Work Skill Development) It is likely to very likely 
(60-95%) that postsecondary technical education and 
training programs will base 80% of their curriculum on 
local customer needs. 

i. (Work Skill Development) it is likely to very likely 
(60-99%) that problem solving ability will be the leading 
employability trait. 

j. (Management) It is likely (60-79%) that management 
functions will lean towards services and support 
coordination. 

3. The committee had an 86% concurrence that: 

a. (Global Issues) It is very likely to likely (80-100%) 
that U.S. companies will be held accountable for any 
ecological damage resulting from production or waste 
disposal. 

b. (Work Skill Development) It is very likely (80-99%) that 
80% of all entry-level personnel will have to be computer 
literate . 

c. (Global Issues) It is likely to very likely (60-99%) 
that the concept of world standards will replace national 
standards as world economy becoiues a realitv by the year 
2000. 

d. (World of Work) It is likely to very likely (60-99%) 
that out of need, corporations will remove the ethnic, 
gender, and disability barriers in order to find skilled 
and/or trainable employees. 

e. (Work Skill Development) It is likely to very likely 
(60-99%) that in education, the understanding of process 
will dominate the current emphasis on product. 

f. . (Government) It is likely (60-79%) that federal tax 

incentives for individuals and corporations will be 
available for formal education and job related training 
and retraining. 

g. (Products and Services: Quality) There is a moderate to 
likely (40-79%) chance that Statistical Process Control 
will be the univt - t l standard (method) . 

h. (Management) There is a moderate to likely (40-79%) 
chance that mid-level managers will be replaced by team 
facilitators who woi.k with the self-management teams in 
any given segment (department of the corporation) . 
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Sampling 

Sampling involves selecting a representative sub-sst of 
people, agencies, or businesses from the total group available. 
Sampling is used for a variety of reasons. It may be very 
expensive to contact the total population or group. Or, it may 
be too time consuming to attempt to contact everyone. The 
alternative is to identify a representative sample and use the 
time anr\ resources .v;«nil.abla to obtain information from members 
of the samt""e„ Research has shown that statistics from a sample 
are oft*:.. .,ori accurate than those from the total population. 
This results from the fact that it is usually possible to do a 
better job o£ contacting each person or organization involved in 
the sample and obtaining the information requested. 

Sampling can be a very sophisticated process. However, the 
basic concepts axe logical and not difficult to apply in everyday 
practice. This module presents these basic concepts and 
identifies a procedure for selecting samples. 

Purpose 

This module is designed to provide the information needed to 
select the appropriate sampling technique and sample size for a 
! study. 

Objectives 
The objectives of this module are to: 

1, Describe basic sampling techniques, and the situations 
in which they can be most effectively applied. 
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2. Provide a process for selecting a valid sample size. 

3. List the typical steps used to select a sample. 

Definitions 

Population 

The population comprises all of the people, organizations, 
or agencies that meet the criteria for the total group. For 
example, if one wanted to measure the career choices of the high 
school seniors in Wisconsin, the population would be all of the 
young men and young women who are high school seniors in 
Wisconsin. For a study of the supervisory training needs of 
employers, the population might be all businesses and industries 
which have one or more supervisors. The list of all of the 
elements or individuals in the population is called the sampling 
frame. The problem or purpose statement should indicate the 
appropriate population. 
Sampling 

A sample is a representative group of individuals or 
elements from the total population, in the example of the 
identification of career choice, a sample of high school seniors 
could be selected. In the study of the need for supervisory 
training, a sample of businesses and industries could be selected 
if this population was large. The sample must be representative 
of the population if the statistics from it are to be used to 
make generalizations or develop conclusions about the population. 
Some basic types of samples are described in the following 
paragraphs . 
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T ypes of Samp les 

Random Samp le 

The process of selecting a random sample assures that each 
person or element in the population has an equal chance of being 
selected, in other words, being selected as a member of a random 
sample is based purely on chance. One random sampling method 
with which we are probably all familiar is placing names in a hat 
and randomly selecting the number of names needed. In this age 
of the computer, software programs are often used to generate a 
random sample from a given population, if the names or 
identification labels for the members of the population are 
stored in a computer data bank, it is relatively easy to obtain 
or write a program which will randomly select members from the 
population and print a list of names, addresses, and/or other 
data on the members of the sample. The names and addresses may 
be printed directly on mailing labels or envelopes to facilitate 
a mail survey. A table of random numbers may also be used to 
select the members of a random sample. A sample set of random 
numbers is given in Appendix A. 
Stratif ied Random Samp le 

In some instances, one may wish to assure that certain 
groups are adequately represented in the sample. For example, if 
a survey of local business and industry was being conducted and 
there were only one or two that were very large, you might miss 
them in selecting a pure random sample, m order to assure that 
one or more of these large industries would appear, one could 

259 



ERIC 257 



stratify the original population by size of the business or 
industry and then select randomly within each of these 
categories. In general, this is the process for selecting a 
stratified random sample. The purpose of the study will dictate 
what categories or groupings (strata) are important to be 
represented in the sample. If there is a possibility that a 
random sampling process might not select enough people or 
elements from that group within the population, a stratified 
random sampling process should be used. 

The first step in selecting a stratified random sample is to 
identify the strata or groupings of interest within the 
population. After these have been identified, a random selection 
of members from within each strata or group can be selected. It 
is possible to change the proportion of members selected from 
each grouping. For instance, if one has a very small number of 
members within any given strata, you may wish to select a 
proportionately larger sample from that group in order to have 
enough data to develop meaningful statistics. If a sample of 
high school seniors was being selected, it would probably not.be 
necessary to stratify the population. This population is 
relatively large and the proportion of males and females is very 
similar. Therefore, it is likely that a sample drawn from this 
population would have an almost equal proportion of males and 
females. In fact, if the proportion of males and females was too 
disproportionate in the sample selected, one would probably 
question the sampling technique used. 
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Sequential Samp ling 

In some situations you have a sampling frame or long list of 
names that identifies your population. Selecting a random sample 
from this list would be a time consuming process and open to 
errors. An alternative process is selecting names from the list 
through the application of a sequential interval K, so that every 
Kth name on the list, following a random start, is included in 
the sample. 

For example, if you need every fifteenth name from the list, 
ranc jmly select a starting point within the first fifteen names. 
Then, count off fifteen names from that point and select that 
person for the sample. This process would be continued until you 
have gone through the entire list of names (sampling frame.) m 
this example, if the tenth name was randomly selected as a 
starting point, you would also be selecting the twenty-fifth, 
fortieth, fifty-fifth, etc. names from the list. The sanple size 
for this process is determined in the same way as you would 
determine the sample size for any of the other sampling 
techniques (See Appendix B) . The. value of interval K is 
determined by dividing the size of the population by the desired 
sample size (N/n=K.) 
Cluster Sample 

When sampling from existing groups, it is often more 
appropriate to select a cluster sample. An existing group could 
be a class section, the people living in a city block, or various 
office/plant sites for a company. In a school setting selecting 
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a random sample can be disruptive to the educational process. 
Also, it makes it difficult to administer the survey instruments. 
When an entire class section is selected, the teacher or 
researcher can administer the forms to all of the students 
involved. 

When doing community surveys, it may be difficult to obtain 
a complete listing of the people in a specific area of a city or 
county. Selecting city blocks or areas can overcome this 
process. 

The process used to select the group or clusters must assure 
that they are representative of the entire population. For 
instance, in a school setting some sections may have more 
stur" nts involved in extra-curricular activities, such as, band 
and athletics. To avoid introducing a bias, the researcher can 
stratify i ha groups or class sections and randomly select from 
the strata. For example, if you had six sections of students in 
each class in high school, these sections could be stratified 
based on those that contain students involved in extra-curricular 
activities and those who are not. There may also be some 
sections that have a cross-section of students. Once the strata 
have been identified, the researcher can randomly select the 
number of sections needed from each strata for the study. Again, 
the number of sections will depend upon the total sample size 
needed. If a sample of size fifty is needed and each class 
section has twenty-five students in it, two sections would be 
selected for the final sample. If there were two strata, one 
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section would be selected from each strata. If there were three 
strata, it may be easier to over sample and select three 
sections . 
Judgmental gajnple. 

Another type of sample sometimes selected is one in which 
the researchers select people or elements from the population 
which they feel are appropriate for the sample needed in the 
study, in some instances, this may be a valid sample, if it is 
necessary to have people who are knowledgeable of a new 
technology or a particular problem area in order to identify 
needs related to the area, then a judgmental sample would be very 
appropriate, if, however, the purpose of the study is to obtain 
data which can be generalized to the total population, then the 
judgmental sample is not appropriate. It is usually difficult to 
prove that ^ judgmental sample, does not have some type of a tias. 
Worse yet, it is usually impossible to identify the exact nature 
and impacts of these biases. 

Procedures for gelegJLjjig a Santp lo 
The following is a general procedure which may be followed 
in selecting a sample. 

1. Review the purpose of the study to identify the 
characteristics of the appropriate population. List these 
characteristics. 

2. Determine if the population is large enough to justify 
using a sampling process, if the population has only eight or 
ten members, it would be more efficient to contact each one. If 
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there are several hundred, then it would be more appropriate and 
efficient to use a sample. 

3. Determine the level of accuracy needed in the sample. 
For example, the level of accuracy needed would increase directly 
as the cost and significance of the impacts of a decision 
increase. Also, determine whether it is necessary to stratify 
the population and sample within each strata. 

4. Using the information from the previous steps and the 
study objectives, determine the sample size to select. If there 
are stratas or sub-groups in the analysis, it will be necessary 
to determine at which level one wishes to specify the accuracy of 
the sample. For example, if it is desired to have a specified 
level of accuracy within each of the strata, then the sample 
sizes would need to be selected for each strata. If it is 
adequate to have a specific level of accuracy for the total 
sample selected, then the sample size would be identified for the 
total group and pro-rated to each of the strata. See Appendix B 
for a sample size chart. 

5. If a ranuom or stratified random sample is being 
selected, use one of the processes identified above to randomly 
select members for the sample. If the members of the population 
are stored in a computer file, it would probably be most 
efficient to have a computer program written which would randomly 
select members from a given population and print out the 
information needed. This program could then be used any time a 
sample is needed. When selecting the sample, it is a good idea to 
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select additional members or names. If it is impossible to 
contact one of the members of the original list, the additional 
names can be used to complete the sample. 

6. Check the characteristics of the sample against the 
characteristics of the total population. For example, if the 
total population is approximately 50% females, the sample should 
also have approximately 50% females, other population 
characteristics suc'.i as age, educational level, and occupation 
may be used for this check. 
A Caution 

Response rates are not equivalent to sample sizes. For 
example, if you need a sample size of 100 and expect a response 
rate of 50%, it is not valid to increase the sample size to 200 
in order to assure 100 responses. The 100 responses would give 
the statistical accuracy needed, but there would be the potential 
of significant bias in the responses. The problem of less than 
100% response rates must be addressed in a different manner. 
Non-respondents must be contacted and data acquired from them. A 
follow-up of a sample of non-respondents is needed to assess the 
nature and magnitude of any biases. The information from this 
non-respondent group is processed separately and contrasted with 
the results from the original respondents. Any discrepancies in 
the response patterns indicate potential biases. 



265 

2B3 



Appendix A - Random Numbers 



* Three Digit Numbers for Populations of 999 or Less. 



* Four Digit Numbers for Populations of 9999 or Less. 
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Three Digit Numbers for Population 
of 999 or Less 

168 998 915 xoo juy jbu b9 190 IPO dftfi 

108 75 467 537 025 231 280 654 349 692 

440 955 936 712 744 346 353 35 553 329 
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Ill 2 ll 364 923 " 2 173 472 49 1 322 
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942 881 437 166 510 786 530 977 564 749 

618 284 116 668 47! ISM nLL III 749 
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Four Digits Numbers for Populations 
of 9999 or Less 

9664 2062 47 5435 4636 9275 5739 3297 8807 2304 
5547 3866 2214 1682 3978 2740 2161 9076 9201 40 
712 5377 4382 5015 6084 5068 3781 3108 4028 6102 
3442 3437 2794 1983 3830 2168 2772 289 9254 4740 
9409 9350 6424 7421 5488 2297 1973 6159 6368 5204 
6310 8194 9270 3540 8628 9401 5222 8648 5417 7843 
9732 3479 2331 1482 5803 4284 6743 5535 370 2498 
5844 2440 2766 8523 2616 4893 1292 6642 9533 9931 
1508 7766 7331 9530 7320 4248 3687 686 3461 8468 
4161 5447 5473 4432 5224 8859 760 9066 6747 313 
1825 7467 8760 1235 282 3791 4855 9614 7728 7025 
2058 9321 7072 8348 4517 9686 5426 4300 5857 8317 
7130 7481 2539 7612 3029 4002 8067 2947 9578 4399 
5391 8000 8872 2232 2345 180 8207 2091 2466 6638 
5245 6224 1041 8818 2396 6505 3718 1021 1498 5967 
2529 5959 7397 1608 1675 7750 6519 7703 2900 340 
4013 6322 7822 6813 594 9297 2038 2552 7634 3330 
3367 2736 6636 7483 4069 2124 6657 9759 437S 2609 
788 6985 2189 1496 279 9740 1282 992 2406 1513 
7856 6854 5667 8843 2778 1687 3414 6634 4370 5843 
4498 382 4071 6397 7034 4519 4822 2165 8877 3602 
9526 245 5892 563 4414 7033 6633 3735 9671 4187 
9498 1882 7161 7520 5983 3937 2592 4277 1101 7684 
6968 1768 9579 9090 9684 7830 3920 1513 3074 1082 
7923 1495 5797 6444 6752 5375 9877 8891 2160 8423 
3745 5903 8946 5728 1139 6590 6249 1550 3190 2954 
4766 1357 8552 6256 8501 8268 3905 8963 7117 3119 
4693 236 2504 4869 3415 436 650 5566 346 7610 
7407 1598 9001 6427 4245 436 6202 7536 7596 9595 
4093 2812 2867 2031 8933 9459 1733 9696 9030 3442 
8791 4457 1640 207 1174 3119 6195 8173 2215 748 
8680 7951 7763 4028 9528 7765 1234 3863 4861 3045 
1000 3956 8415 5659 1201 9555 9020 5745 4584 8123 
5944 6632 3453 1117 1972 795 2329 3615 1548 2325 
3417 6005 8832 8891 3461 9445 8012 3641 8095 4333 
5716 5027 7280 8483 2864 25 2913 924-1 813 3702 
7210 1268 4130 5617 7957 9156 9837 10^5 4212 1691 
480 2280 3573 1885 5851 3628 9578 4553 5731 9498 
5354 637 3464 4608 7099 4996 7740 1158 8490 1883 
7406 6136 4611 9819 1662 9200 1501 5103 5479 9105 
4907 652 7588 4145 9341 4409 5878 9578 6035 9023 
2556 2498 5881 2038 8895 2132 964 3286 6940 1024 
5762 3726 2988 72 6627 7431 5209 790U 5626 5953 
5233 2518 2341 1771 1064 746 4723 5064 6148 8374 
7595 5496 6953 1522 4889 8265 8893 7166 9224 1108 
7952 8309 9904 6495 2841 6755 7947 730 33 3960 
4219 9170 1531 7930 4288 4290 2695 1628 1067 807 
4990 5871 3868 9581 5981 7169 9937 9518 6434 8491 
2074 4189 3860 8416 3434 7381 3340 1646 8099 9308 
9991 1336 3509 2688 2896 1231 8363 5420 7861 3054 
7062 4064 1827 2140 9305 9855 5294 1757 924 6464 
590 4108 9551 6225 3339 3469 6215 9045 1955 4427 
1457 5728 7532 5708 2068 4909 3947 7533 2026 340 
5534 8007 8742 2729 7174 1472 3202 7866 9137 9323 
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6030 
681 
1494 
3633 
7992 
3995 
5449 
12 
144 5 
7932 
8819 
448 
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6851 
6507 
5364 
2626 
5614 
368 0 
6730 
9413 
4017 
4421 
4685 
8116 
602 
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3533 
2764 
5840 
53 
7112 
9574 
4687 
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8415 
8519 
8783 
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6443 
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5691 
6695 
1867 
6490 
6564 
976 
8319 
8241 
9975 
9956 
3009 
9439 
635 
8401 
781 
1178 
7536 
5535 
7157 
3148 
2220 
3140 
5040 
8181 
8909 
539 
3288 
2920 
6518 
6453 
7160 
5656 
2547 
7727 
646 
1004 
8834 
4942 
658 
1442 
9199 
4889 
3113 
5727 
8954 
9928 
875 
4191 
6286 
4217 
3584 
908 
5524 
6821 



9968 
960 
2186 
9085 
3004 
751 
5767 
9817 
8167 
4579 
4445 
9481 
4863 
4010 
6637 
6737 
8074 
997G 
5114 
7995 
7008 
9366 
5342 
1220 
8131 
1080 
8199 
9118 
2565 
1344 
7058 
1694 
7866 
8483 
9259 
8187 
4180 
7037 
1947 
8376 
4765 
1260 
2283 
3266 
897 
7984 
2039 
5145 
1147 
8874 
6559 
7018 
5031 
2040 



905 
7674 
8021 
9320 
3297 
5115 
6123 
6223 
155 
6138 
7226 
2248 
6204 
6329 
1653 
7573 
4626 
9238 
5443 
7516 
5794 
8707 
5015 
8303 
2874 
3067 
8716 
4701 
4309 
1210 
5235 
9531 
3171 
544 
8899 
5052 
3395 
6811 
1838 
6228 
2881 
5791 
3031 
5859 
3478 
6873 
4222 
1599 
7836 
8505 
2714 
4836 
3139 
5520 
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Sample Sizes 

The Sample Size Chart can be used to determine a sample 
size. Find the size of the population in the column labeled, 
Population Size and read the sample size in the next Sample Size 
column to the right. These sample sizes will provide data with + 
or - 5% accuracy in 95 samples out of 100. (I.E., if one found 
that 50% of a sample responded yes to an item, it is very likely 
that 45% to 55% of the population feels the same way.) 
Selecting the Sample Size Select the sample size to give the 
level of accuracy needed for your study. For example, if you are 
surveying a population of 1000 students and want to have + or - 

* 

5% accuracy, select a sample size of 278. If, however, you want 
+ or - 5% accuracy for the statistics for the responses from 
males and females, you would have to base the sample sizes on the 
number of males and females. For the purposes of this 
illustration, let us assume that there are 500 males and 500 
females in the population. To obtain + or - 5% accuracy for each 
of these groups, the sample sizes would be 217 malas and 217 
females. The total sample size of 434 would give statistics with 
more than + or - 5% accuracy for the total group. However, the 
study would also be more expensive since more people were 
involved in the survey. 

You will need to decide the level (male/ female, total group, 
etc.) at which the most critical decisions are made and select 
the sample sizes at that level. 



278 



Sample Size Chart 



Population Size# 


Sample Size 


Accuracy* 


Population Size# 


Sample Size 


Accuracy 


25 


24 


+ 5% 


1000 


278 


+ 5% 


50 


44 


+ 5% 


1500 


306 


+ 5% 


75 


63 


+ 5% 


2000 


322 


+ 5% 


100 


80 


+ 5% 


5000 


i 

357 


+ 5% 


150 


108 


+ 5% 


10,000 


370 


+ 5% 


200 


133 


+ 5% 


50,000 


381 


+ 5% 


250 


152 


+ 5% 


300,000 


384 


+ 5% 


300 


169 


+ 5% 


Very 


250 


+ 7% 


400 


196 


+ 5% 


Large 


1000 


+ 4% 


500 


217 


+ 5% 


Populations 


1500 


+ 3% 


600 


234 


+ 5% 








7 0 


245 


+ 5% 








800 


260 


+ 5% 








900 


269 


+ 5% 









ft For populations between the sizes listed, interpolate from the sample sizes given. 
* For values near 50% (this is region of largest error) . 
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Working with Small Groups 
Introduction 

Many individuals fail to participate in large group 
discussion because they t^e intimidated by the group's sheer size 
and diverse background, or because the high number of group 
members does not provide opportunity for everyone to sp^ak. 
Small group interaction often reduces shy individuals' 
inhibitions and allows participation by all group members, it is 
advised that small group sizes be kept below eight people. 

Small group structure can be rigid or flexible, introduce 
new members or strengthen existing bonds, facilitate new and 
creative ideas or dissect a problem and determine a solution. 
However used, small groups can be an effective tool for trainees 
or teachers. 
Selection 

Before we begin with the role of group facilitators and 
workings of small groups, it may be beneficial to briefly explain 
' ow to choose the members of small groups. Perhaps the most f ' 
critical aspect in selecting group members is to clearly define 
the purpose of the groups and the outcome desired. 

If the purpose of the group is to predict the future skill 
need*; for entry level work force members in five years, it would 
be advisable to use employers as group members. 

If the group's purpose is to decide if punishment of school 
violations is being distributed evenly for athletes and non- 
athletes, a group comprising student athletes, student non- 
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athletes, and administration officials would be advisable. 

If an instructor simply wants to increase class discussions 
through small group interaction, a random grouping may be best. 
It should be noted that if small group discussion is used often 
in a classroom, it is advisable to rotate, group members regularly 
to maximize student to student interaction. 

For more information on how to choose group members see 
modules on Delphi Survey and Sampling. 

Purpose 

The purpose of this module is to explain the role of the 
facilitator in small group discussion and to provide examples of 
various small group methods. 

Objectives 

The objectives of this module are to: 

1. Describe the role of the facilitator in small groups. 

2. Provide exercises to explore how groups operate. 

3. Provide examples of various group methodologies. 

Facilitator's Role In Small Group Discussion 
During group work or discussion instructors perform the role 
of facilitator. While the role of the facilitator varies 
depending on the group activity, it remains the vital key to the 
group's effectiveness. The facilitator plans, organizes, and 
directs the group's activities. Though the students provide 
information and discussion, it is the facilitator who maintains a 
comfortable atmosphere conducive to open yet controlled 
discussion. The facilitator will: 
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X. Introduce the objectives of the group and explain the 
parameters for discussions/activities. 

2. Guide/lead the group discussion/activity. 

a. Keep on target 

b. Involve all people 

c. Quiet people who want to dominate 

3, Summarize the group's findings. 

It is important for a facilitator to understand how groups 
work, know what activities work best for various group 
objectives, and to be well organized yet sensitive and flexible 
enough to adapt to changes in the group or objectives. 

Phas es of Group Interaction 

Any group, regardless of size, goes through three distinct 
phases as it moves toward accomplishing its goal. Although these 
phases may be mismanaged by unskilled group leadership, they are 
clear and distinct. The three phases are Clarification, 
Evaluation, and Decision. 
C larification 

In this phases facts and opinions concerning the situation 
are presented and leadership roles begin to emerge within the 
group. This phase should represent a minimum of 50% of the group 
time. Two things must be accomplished in this phase. 

1. Group objectives must be clearly stated. 

2 . Roles to be assumed by each group member must be 
tried out, accepted by others, and then stabilized. 
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During the evaluation phase group menders work on the 
goals/ objectives discussed in the clarification phase. 
Decision 

During the decision phase the group selects the best 
solution or final decision based on the group's activities. 

Learning How Groups Work Through Experiential Learning 

Experiencing the processes and sometimes frustrations of how 
groups operate can often be the first step in successful group 
work. Cog's Ladder (see University Associates 1974 Annual for 
Facilitators, Trainers, and Consultants) is an excellent exercise 
that explores the factors which distinguish process from content 
in group interaction. Each group will go through distinct steps 
or phases in the process of interaction. How well the 
facilitator understands and directs these phases often will 
dictate the quality of content the group develops. 

Group consensus seeking exercises are also excellent 
beginnings for group problem solving activities. Examples of 
group consensus seeking exercises are contained in modules #307- 
Group Problem Solving: An Experiential Learning Exercise, and 
#308 - Group Problem Solving: Lost at Sea, A Group Consensus 
Exercise (these modules were adapted with permission from 
University Associates Annuals for Facilitators, Trainers, and 
Consultants) . 

Fish bowl type exercises provide participants an opportunity 
to observe and experience the group process. In this exercise 
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one half the members are seated in a circle and given a topic to 
discuss. The other half are positioned in a circle surrounding 
the first and observe. The roles are then switched so each group 
discusses and observes. After this process the facilitator 
directs a group discussion on what was experienced and observed. 
Types of Small G roup Methods of Instruction 

Which ever the small group methods chosen it is important to 
provide a means of recording the ideas or findings of the group. 
Sheets of paper, chalk boards, white boards are all acceptable; 
however perhaps the most common and flexible is a flip chart. 
Flip charts are cheap, easy to transport, can be preprinted with 
questions or guidelines and are easy to use. 
Buzz Groups 

Procedure . 

Large group is broken down into smaller groups of five to 
ten people. Each group openly discusses their assigned topic 
then reports to the large group. A group leader is chosen to 
direct group discussion and to report the findings. 

Advantages. 

1. Involves everyone. 

2. Reduces intimidation of speaking to large group. 

3. Small number facilitates communication between members. 
Brainstorming 

Procedure . 

Group members are given a topic and are asked to vocalize 
any or all ideas about that topic. It is important to remember 
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three things. 

1. No judgments, or "all comments are good comments." 

2. Quantity of ideas/ comments are desired. 

3. Creativity is desired. 
Advantages . 

1. Because all comments are good, intimidation is low. 

2. Produces a large guantity of information. 

3. Facilitates creativity. 
Nominal Group Technique (NGT) 

Procedure . 

La:-ge groups are divided into groups of 5-8 people and given 
a topic or problem. Individuals are given time to think to 
theraselv* .. about the topic. A group leader then directs: 

1. In a round robin fashion, the listing of all ideas from 
each person. This is done without discussion. 

2. Discussion within each group to clarify or expand all 
items on the flip chart. 

3. Each individual to vote for the top five or ten items. 

4. The group to make a priority list from those items 
(priority list length will vary by topic) . 

5. Each group to report it's findings to large group. 
Advantages . 

1. Good tool for problem solving. 

2. Provides both qualitative and quantitative information. 
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Phillips 66 

Procedure . 

Uses the .^ame procedures as Buzz groups except each group 
has six members and discusses each topic for only six minutes. 
Advantages . 

X. Involves everyone. 

2. Facilitates communication. 

3. Enforced time constraints keep members on the topic. 
Experiential Learning 

There are numerous experiential learning exercises covering 
a wide range of topics. Exercise lengths may vary from five 
minutes to two hours and are an effective way to involve group 
members. For a more complete explanation and list of exercises 
see University Associates' Annuals for Facilitators. Trainees, 
and Consultants 

Advantages . 

1. Fun change of pace. 

2. Involves all group members. 

3. Applies what is learned. 
Case Studies 

Procedure . 

Group members are given an actual incident that did or could 
happen in the related field. The small groups discuss possible 
problems, solutions and ramifications of the incident. 
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Advantages . 

1. Used "real life" situations. 

2. Members can call on past or present experiences to 
discuss the incident. 

3. Incident example may be structured or modified to fit 
group and objectives of the training tor educational 
activity. 

Focus Group s 

Focus groups are a popular means of acquiring a qualitative 
set of responses from a group of people familiar with the topic 
being discussed. The group consists of eight to twelve people 
working with a moderator to express opinions and attitudes and to 
discuss the specific topic that all group participants are 
familiar with. Sessions usually run from one to two hours. 

Procedures . 

1. Initial client meeting 

2. Development of the interview guide 

3. Second client meeting 

4. Conduct of the groups 

5. Transcription of the proceedings 

6. Internalization 

7. Organization and writing 

8. Presentation of the findings 
Advantages 

1. Provide easy, fairly reliable access to the ideas and 
attitudes of a work group. 
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Provides good data to be put into questionnaires for 
quantitative study. 

For a wore in-depth look at focus groups, see Zemke. 
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Introduction 

The age-old questions previously asked by educators in the 
past, who were concerned with education for work, remain today. 
First, what do students need to know to be successful. Second, 
what skills or competencies do students, as learners need to 
acquire to be competitive in the workplace. Although the 
questions remain the same, the answers may vary from generation 
to generation, from year to year and from geographic location to 
geographic location in response to a variety of convoluting 
factors . 

To keep curriculum current with locally determined needs, 
schools must develop a network of workplace needs. One can draw 
upon reviews of relative research, state and federal guidelines, 
follow-up studies of local/regional graduates, surveys of 
business and industry, and interviews of local business and 
industry representatives, as excellent sources. This is 
especially true when information from the various sources are 
combined and compared to attain up-dated information upon which 
to base decisions. 

One of the goals of the High-Technology Training Model 
Project was the development of a Necessary Skills Matrix. The 
intent was to identify those skills necessary to acquire 
employment and be a competent contributor within the workplace. 
Data was collected from; (a) the Wisconsin Department of Public 
Instruction's Education for Employment Program, (b) five year 
follow-up studies of high school graduates conducted as part of 
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the High-Technology Training Model Project, (c) interviews of 

*r 

local business and industry representatives, and (d) business and 
industry surveys. 

Purpose 

The purpose of this module is to provide an example of how 
to conduct a necessary skills study and to report the findings of 
the High Technology Training Model Projects' Skills/Competencies 
study . 

Obi ectives 
The objectives of this module are to: 

1. Provide an example of how to conduct a necessary skills 
study . 

2. show the results of the High technology's study on 
necessary skills/ competencies. 

M r thodology 

Data was gathered through project generated industry 
interviews and surveys, and high school five year follow-up 
studies. This data was pooled with skills/competencies lists 
published by Wisconsin's Department of Public Instruction (DPI). 

A base list of skills for the matrix was formed by combining 
lists of DPI's employability and basic skills contained in the 
Wisconsin DPI publication, Education for Employment, where 
duplication of skills was found, only one entry was made. Skills 
from the other studies were then either categorized within the 
base list, or added to the list. 

It must be noted that these various lists may not be 
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directly comparable on a one-to-one basis. It is the composite 
of these lists that is important. One must also accept the fact 
that the various studies were designed to identify local 
requirements in response to different needs. 
Information sources 

The following is a summary of each of the studies used in 
the development of the Necessary Skills Matrix. 

DPI Basic Skills . 

This basic skills list was taken from Education For 
Employment, published by the Wisconsin Department of Public 
Instruction (DPI) in 1988. The Department of Public Instruction's 
stated purpose for their lasic skills list was to identify those 
skills that all K-12 students must have to; (a) advance to 
postsecondary education, (b) promote a successful transition from 
high school jo the world of work, and (c) master the lifelong 
skill of learning how to learn. 

The implication of this basic skills list can be far 
reaching if properly used. The Department of Public Instructions 
goal is to establish staff development activities that are 
designed to assist K-12 teachers and business/community 
representatives integrate; (a) basic skills into the vocational 
curriculum, (b) world of work examples and competencies into the 
general education curriculum, and (c) activities across grade 
levels and disciplines with special regard for student 
populations with unique learning needs. 

In conclusion, DPI's basic skills list was designed to serve 
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as a base or minimum standard by which K-12 teachers can 
integrate relevant skills/competencies into all their curriculum. 
The skills listed are those considered necessary for the 
successful transition into postsecondary education or the world 
of work, and to develop lifelong learning skills. 
DPI Emplo/abilitv Skills . 

Initiated in response to a growing concern over the number 
of youth who fail to make a successful transition from school to 
the work world, Wisconsin's Department of Public Instruction 
joined forces with Parker Pen Foundation of Janesville, WI. The 
Parker Project's first phase provided the necessary information 
base required to assess the school's capacity to meet the needs 
of business, students, and society. This information base was 
also used to project future business needs. The Wisconsin's 
Education for Employment concept grew out of the Parker Project's 
work. 

Teaching the skills necessary for students to seek, attain 
and maintain jobs is a primary objective of Wisconsin's Education 
for Employment . By incorporating employability skills and 
standards, such as punctuality, responsibility and general work 
ethics, into the schools' curriculums, students will be better 
prepared to make a successful transition into the work place. 

Too many high school students do not understand the 
importance placed on punctuality, taking responsibility for 
oneself and actions, and working together; and therefore, are ill 
prepared for the work place and its demands. By incorporating 
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these skills/attitudes into the schools, the students will be 
better able to perform on the job. 
B & I Surveys . 

The statewide business and industry survey was conducted by 
project staff and based on a random sample of 285 companies drawn 
from a random list of 12,000 businesses and industries generated 
by the Wisconsin Department of Industry, Labor, and Human 
Relations (DILHR) . it should be noted that the DILHR sample did 
not include representatives of the hotel and tourism industries. 
The purpose of the survey was to identify; (a) who within 
companies is responsible for training, (b) the type of training 
currently being done, and (c) that which is planned for the near 
future . 

Although, all the collected data was not directly applicable 
to the Necessa ry skills Matrix r the comments of the 67 
respondents surveyed indicated that companies were highly 
involved in training in such areas as interpersonal relations, 
problem solving/reasoning, job seeking and getting, work ethics, 
computer skills, and a wider, stronger base of specific technical 
knowledge. 

It seems imperative that industry be surveyed on a regular 
basis and that feedback be incorporated into the institution's or 
district's curriculum development/ evaluation process. 

B & I Interviews . 

During the summer and fall of 1989, participating teachers 
were asked to become more familiar with the technological changes 
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taking place in the businesses and industries of their local 
communities. Participating teachers conducted interviews of 103 
local companies using a project staff prepared set of questions. 
Open-ended questions were used to identify any new technologies 
that companies had implemented in the last two years and the 
resulting new skills and competencies needed to operate or use 
those technologies. Also asked was what new technologies 
companies seriously considered implementing within the next 3-5 
years, 

A second part of the interview of local businesses and 
industries conducted by participating teachers identified new 
skills and competencies current employees and future entry level 
employees need. 

Of particular interest was the strong emphasis industry 
placed on the basic skills: reading, writing , speaking and 
listening , and both basic and advanced math. The results also 
showed a heavy need for interpersonal relations and problem 
solving skills. It should be noted that representatives of local 
businesses and industries indicated that computer skills are 
every bit as important as the traditional basic skills. 

The participating educators 1 interviews of individuals 
representing local industries spawned two excellent outcomes. 
First , a list of skills/competencies industry perceives students 
need to learn. Second , the interviews provided an educational 
experience for the teachers which in turn provided them with new 
content and applications for their courses, 
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Five Year Follow-up 

Five year follow-up studies of graduates were conducted by 
each of the secondary and postsecondary schools participating in 
the High-Technology Training Model Project. The goal was to 
obtain feedback regarding employment status and high school 
preparation. The surveys also requested input regarding the 
skills or knowledge students had to acquire since graduation, as 
well as those they anticipate they will have to acquire in the 
near future to keep up with changes on the job. 

General job, general preparation, English and math skills 
were rated as being of greatest value in high school preparation. 
Responses to open-ended questions asking what present and future 
skills will be needed in the work place indicated a significant 
need for computer, interpersonal relations, supervisory, and 
technical literacy skills. 

Separate modules have been developed and are included in 
this handbook covering each of the research activities conducted 
as part of the High-Technology Training Model Project. 

Findings of the Study 

This study brought a split image of the needed skills into 
focus. The Wisconsin department of Public Instruction's basic 
and employability skills address the skills considered necessary 
to make the successful transition to postsecondary education, 
achieve employment, and be prepared for lifelong learning. 

Business and industry surveys and interviews along with the 
five year follow-up studies of high school graduates identified 
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those skills the respondents considered necessary to become and 
remain competitive in the wor olace. 

Appendix A shows a consolidated matrix listii^ only items 
which were mentioned in at least two of the studies. 
Interpersonal relations and problem solving were the only items 
mentioned in all studies. Computer skills and work ethics were 
noted in four of the five studies; and communication, advanced 
math, and electronics were identified in three of the five. 

It should be noted that va vying applications of technology 
backgrounds were consistently identified by business and industry 
as skills needed by its work force. 

Significance of the Necessary Skills Matrix 
This study has shown there is a significant need for 
teaching our students the following skills/competencies: 



1. 


oral and written communication, 


2. 


basic and advanced math, 


3. 


interpersonal relations, 


4. 


problem solving, 


5. 


work ethics, 


6. 


basic through advanced computer, 


7. 


electronics and, 


8. 


strong technical literacy base. 




Another significant finding in this study was the 



thoroughness of the business and industry interviews. Educators 
talking directly with business and industry experts provide an 
opportunity for clear, probing questions about skills needs to be 
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answered by those v'o do the hiring and see the needs on a daily 
basis. Surveys ad follow-ups are fine sources of information, 
but nothing beats a one-on- ne discussion to solve a problem. 



ERIC 



302 



3 J 1 



APPENDIX A 
Necessary Skills Matrix 
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Articulation: Helping Students 
Transition from Secondary to Postsecondary 

Institutions 

Introduction 

Within the United States, academic education, general 
education, and vocational education are generally considered to 
be based on separate and distinct bodies of knowledge and skills. 
Feldman (1984), suggests that the educational system in the 
United states has traditionally fostered this competitive 
relationship and the perceived need to make these educational 
distinctions has precluded the integration of thoughts and 
practice concerning education. 

Recent discussions addressing ways to improve the quality 
of education suggest the integration of academic and general 
aspects of education with the vocational aspects. Plihal, 
Johnson, Bentley, Morgaine, and Liang (in press), suggest that 
any attempt to integrate the vocational, academic, and general 
aspects of the educational system in the United States will 
require changes in the philosophy and organization of schools as 
well as the instructional methods employed. Integrations of the 
various aspects of the educational system will not, in and of it 
self, provide the necessary motivation for those students who 
lack a clear vision of an attainable future. 

The educational system within the United States was not 
designed as a total system which allows students to complete 
their formal education at a single institution. Students move 
from exploratory programs at the middle school/ junior high 
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school, through the secondary/high schools, and then on to 
technical colleges, junior colleges, and/or four year co^eges 
and universities. The entire process of transitioning students 
from one level of education to another, or between one 
institution and another, is what Selman and Wilmoth (1989) define 
as Articulation . 

The question addressed in this module is, how does 
education, as a system, transition students from secondary to 
postsecondary institutions? It should be noted that the 
discussion in this paper is focused on the articulation programs 
designed to help students make the smooth transition from 
secondary to postsecondary institutions. 

Purpose 

The purpose of this module is to define selected forms of 
articulation which can be used to facilitate student transition 
from secondary to postsecondary educational institutions. 

Objectives 

The objectives of this module are to: 
1. Provide a general description of several articulation terms 
and forms of articulation which include: (a) barrier removal 
programs, (b) enrichment programs, (c) shared facilities, (d) 
combined enrollment, (e) contracted services, (f) transfer of 
credits, (g) time-shortened programs, (h) advance placement, (i) 
one-plus-one, (j) advanced skills programs, (k) core curriculum 
or pre-technology, and (1) vocational technical two-plus-two 
preparation. 
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2. To provide general information and examples pertaining to 
the development and administration of selected forms of 
articulation which help students 1 transition from secondary to 
postsecondary educational institutions. 

Stipulat inq the Meaning of Terms 
This module on the articulation programs began with efforts 
to make sense of language and concepts central to the topic. The 
terms used to define the various articulation programs are used 
with a variety of meanings by different people. Additionally , 
when trying to review literature to make sense of these 
articulation programs , one encounters related terms such as 
curriculum integration, academic education, general education, 
liberal education, and vocational education. To add to the 
confusion, some terms take on different meaning at the secondary 
and postsecondary education levels. 
Articulation 

When applied to programs between secondary and postsecondary 
institutions, Long, Warmbrod, Faddis, and Lerner (1986) describe 
this application of the term articulation as "a planned process 
linking two or more educational systems within a community to 
help students make a smooth transition from one level of 
institution to another without experiencing delays or loss of 
credit" (p.l). 

In vocational education, articulation has been defined in a 
number of ways but the central theme is: "to eliminate as much 
as possible, unnecessary duplication of training across the 
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levels" (long et al., 1986, p.2) . 
Curriculum Integration 

Curriculum integration as defined by Plihal et al. (in 
press) , refers to the process of combining the various parts of a 
student's educational experience to form a single, more 
encompassing, experience. 

As seen by Plinal et al. (in press), horizontal curriculum 
integration occurs when the curriculum enables a student to learn 
various principles (in one course) and develop a full 
understanding of those same principles by applying them in 
another course at the same grade level. Vertical curriculum 
integration, on the other, purposefully connects parts of a 
student's educational experience from one level to another. 
Academic Education 

Academic education at the secondary school level refers to 
education organized around generally accepted disciplines and is 
usually limited to the knowledge areas of English, foreign 
languages, history, economics, mathematics, and the sciences 
(Good, 1973). At the secondary level, all students are required 
to take a minimum number of courser in a prescribed academic area 
as a prerequisite for graduation. Students planning to attend 
college are usually in a college preparatory program (track) and 
take more than the minimum number of academic courses required to 
meet college entrance requirements and expectations (Plihal 
et.al. , in press) . 
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General Education 

Shapiro (1986), describes general education as unfocused and 
the dominant track in most high school curriculums. According to 
McConnell (1952), general education grew out of concerns that at 
the college and university level, educations were becoming overly 
specialized while liberal education has become technical, narrow 
and irrelevant to human concerns. As an out growth of that 
concern, general education at the secondary level, was designed 
to develop the common core of knowledge, skills and values 
considered necessary for students to deal with the personal and 
social problems expected to confront them in a democratic society 
(Purves, 1988; Faust, 1950). To some the concept of general 
education includes both academic and vocational courses (Plihal 
et al., in press). Parnell (1985), referring to the general 
education as the academic and vocational desert in the American 
education system, described it as being made up of a combination 
of general, remedial and personal/hobby courses. As Plihal et 
al., (in press) noted, "It is unfortunate that the term 'general' 
often is used at the secondary level to refer to a track that is 
neither academic (college-preparatory) nor vocational— a 
curriculum that reportedly prepares students least well for life 
after high school" (p. 7). 
Liberal Education 

Although sometime used synonymously with general education, 
liberal education has a different meaning. Frequently is liberal 
education used to describe college level education which prepares 
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students for living rather than preparing them to earn a living. 
Liberal education is education with intrinsic rather than 
extrinsic worth; education intended to be appreciated rather than 
applied. The seven liberal arts traditionally comprising a 
liberal education are: grammar, rhetoric, logic or dialectic, 
arithmetic, geometry, astronomy, and music (Plihal et al., in 
press) . 

As Plihal et al. (in press) noted, the distinction for 
categorizing selected subjects — mathematics and the sciences — as 
part of an academic or liberal curriculum, is dependent upon 
their intended use and the intrinsic or extrinsic value which is 
attached to them. 
Vocational Education 

Vocational education is most commonly used in the broad 
sense to refer to education which prepares people for gainful 
occupations (Prosser & Quigley, 1949) . Variations of this 
definition refer to education that prepares people for work 
(Roberts, 1965) or work and family role (Copa, Plihal, Scholl, 
Ernst, Rehm, & Copa, 1985). In a more focused perspective, 
vocational education can be defined as a specific set of skills, 
attitudes and understandings a particular group of students are 
to develop. The National Commission on Secondary Vocational 
Education (1984), sets the following as the five areas vocational 
education should develop in students: (1) personal technical 
literacy, (2) communication and computational skills and 
technical literacy, (3) employability skills, (4) broad and 
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specific occupational skills and knowledge, and (5) foundations 
for career planning and lifelong learning" (p. 3). 

Vocational education is usually offered through the 
following subject areas or subfields: (a) agriculture, (b) 
business, (c) consumei home economics, (d) industrial arts, (e) 
marketing, (f) occupational health, (g) occupational home 
economics, (h) technology, and (i) trades and industry (T & I) 
(Plihal et al., in press). Copa and Bentley (in press), identify 
career education as a general subfield of vocational education, 
(cited in Plihal et al., in press* . 

All too often at the secondary level, vocational education 
programs are not viewed as the link to the next educational 
level. If vocational education is reduced to "a euphemism for 
the handling of students with behavioral problems" (shapiro, 
1986, p. 91), one can not help but wonder if many of these 
difficult students behave the way they do because they see no 
future in their educational — academic and vocational — programs. 
Unable to visualize the relationship between their high school 
experiences and the knowledge, skills, and attitudes needed to 
take on the challenges which lie ahead. 

Articulation Programs 

Articulation programs are known by various names. Warmbrod 
and Long (1936) suggest that the articulation programs between 
secondary and postsecondary two year institutions, make high 
school students, embarked on attainable vocational technical 
career programs which span grades ll through 14, feel like they 
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are college-bound as they catch views of new opportunities and a 
better, possible future. However, not all articulation programs, 
are the same. The term articulation has been loosely applied to 
a broad range of programs. To add even more confusion, selected 
terms used to identify specific articulation programs have also 
been used as collective titles for groups of programs. The 
confusion arises when the titles used to group articulation 
programs do not adequately describe all programs within the 
group. Each articulation program is unique and meets a specific 
educational need. Educators at all levels and in all disciplines 
should be familiar with the more common term used to identify 
articulation programs and their meaning. 
Barrier Removal Programs 

Not all so called articulation programs are specifically 
designed to meet the Long et al. (1986) definition of helping 
students' transition from one institutional level to another. 
Articulation efforts which simply link two or more institutions 
to remove educational barriers and thereby benefit the student, 
are not new. Manley (1970) defined a well articulated 
educational program as one which provides its students the 
opportunities to develop to their highest potential in attaining 
educational as well as career objectives. The first four 
articulation programs meet Manley 's definition of articulation by 
removing educational barriers. 

Enrichment Programs . 

Enrichment programs are usually developed by postsecondary 
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institutions in response from one or more local school districts. 
Program arrangements are generally quite simple, and require the 
student to obtain a letter from the high school principal stating 
tha 4 - their attendance at the postsecondary institution will not 
intei "ere with the student's secondary school work. As noted by 
Lerner (1987) , students can receive dual high school and college 
credit. Students receive their high school diploma in addition 
to earning credit toward a college degree for participating on a 
full-time or part-time basis, during the regular academic year as 
well as during summer terms. For most enrichment programs, 
postsecondary credits are held in escrow for later application 
toward a program at the participating institution. In some 
cases, the earned credits may be transferred to other 
postsecondary institutions. 

Enrichment programs are typically oriented toward 
accelerated students entering their -senior year who plan to 
attend a postsecondary school after graduation. Lerner (1987) 
noted that enrichment programs can permit the participation of 
selected 10th and 11th graders. In addition to the regular 
postsecondary curriculum, programs could also include non-credit 
seminars, conferences, and workshops as summer enrichment 
activities. 

Shared Facilities . 

Having two types of schools in close proximity has the 
advantage of sharing selected facilities. Lerner (1987) noted 
that this may be particularly true when the institutions involved 
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are required or wouM like to provide services or acquire 
expensive, sophisticated equipment and/or facilities that neither 
institution could provide on their own. Fully accepting the 
possibility of periodic scheduling problems; shared facilities 
are cost effective, represent the best use of equipment, and can 
greatly enhance the recruitment efforts. 

Shared facilities typically include athletic facilities, 
dining facilities, libraries, and specialized classrooms and 
laboratories. 

Combined Enrollments . 

Although not as obvious between secondary and postsecondary 
institutions, there are advantages of having students from one or 
more high schools (public and/ or private) enrolled in the same 
class. This is particularly beneficial when neither of the 
participating institutions have a sufficient number of students 
to justify offering a particular course. Dual enrollment has a 
great deal of merit from an economic, as well as an educational 
standpoint. The possibility of a secondary institution using 
postsecondary facilities, noted by Lerner (1987), takes 
advantage of the best equipment available while making possible 
the use of additional facilities. As an alternative, consider the 
possibility of distant learning and the technological 
advancements which are available. The opportunities to combine 
enrollments quickly take on a different appearance. The 
development of magnet school for specific courses or programs, 
are a variation of the combined enrollment concept. 
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Contract ed Services . 

Contractual agreements to offer classes at other 
institutions has a multitude of possibilities both horizontally 
as well as vertically. Again, the technological advancements in 
educational delivery systems provide a multitude of 
opportunities. The state of Wisconsin is divided into eleven 
Cooperative Educational Services Agencies (CESA) areas. Each 
CESA provides, among other things, contracted services which the 
individual, local educational agencies could not provide or 
justify, in specific instances, it may be desirable to have 
postsecondary institutions contract to provide certain advance 
courses taught in the high school. Contracting as noted by 
Lerner (1987), represents an alternative source of instructional 
faculty, it also provides students some exposure to alternate 
forms of higher education. 
Credit Transfer 

Few examples could be found of cooperative efforts between 
public secondary institutions and public postsecondary vocational 
technical institutions for direct transfer of credits. Such 
agreements, which can serve as models, do exist between public 
secondary institutions and proprietary occupational institutions 
as well as community/ junior colleges. 

Lerner (1987) noted, that students who complete properly 
developed and managed secondary vocational education program, 
acquire very specific skills and knowledge in preparation for 
employment. Some community colleges grant limited college 
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credits for these achievements. The number of credits 
transferred depends on the degree to which the stated secondary 
competencies can be integrated into the postsecondary curriculum. 
Lerner (1987) indicated that 3-18 hours of credit can usually be 
granted at the postsecondary for secondary school preparation, 
depending on the particular discipline in question. The awarding 
of advance credits generally follow one of two scenarios. 

1. Students who earn "A" grades in approved secondary courses 
receive the advance credit (s) directly. 

2. Students who receive "B" grades in approved secondary 
courses, gain advanced credits through a credit by examination 
procedure . 

In both cases, applicants for the tre. : of credits must 
meet the college and program requi memento id matriculate. 

This process of transferring credits for authorized 
secondary courses differ fro*a the advance placement (the next 
articulation program) , in that the credits are accepted prior to 
commencement of the postsecondary course of study and are awarded 
as soon as the student matriculates. Spencer ian College, as 
noted by Lerner (1987), allows the transfer of 16 secondary 
credits (4 classes) for courses included in Figure 1. 
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Figure 1. Typical secondary courses for which Spencerian College 

allow up to 16 credits to be transferred for qualified, 

matriculating students. 

English Math 

Typing Shorthand - Gregg 

Business Law Business Correspondence 

(Figure cont.) 
Secretarial Accounting Accounting I 

Human Relations Filing and Record Management 

Anatomy and Physiology Medical Terminology 

Salesmanship Business Communication 

Merchandise Math Consumer Economics 

Source: Lerner, (1987) . 

If a student from a qualified high school earned "A" or "B" 

grades in approved courses and could transfer 16 credits (quarter 

or semester) into a Minnesota or Wisconsin state 

college/university, it would equate to approximately $1,000 in 

tuition alone. For private postsecondary institutions it could 

be even more. This is a recruiting tool technical colleges can 

not currently touch. Lerner (1987) appropriately points out 

that: 

Traditionally, two-year postsecondary teachers have 
been reluctant to accept secondary learning experiences for 
college credit, yet these same teachers expect four-year 
colleges to accept their students' accomplishments without 
question, often, members of the faculty believe the 
students are not qualified and will not do well at the 
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post secondary level (p. 8). 

It would appear that the technical colleges Lerner (1987) is 
referring to, may be creating barriers to what might be 
reasonable considerations on their part, only to lose more than 
they gain. 

Time-Shortened Programs 

Most articulation programs are designed to facilitate 
advance placement in postsecondary programs for students who have 
mastered fundamental competencies in high school. Collectively 
time-shortened programs are designed to allow the student to 
complete the postsecondary phase faster while saving the student 
the tuition equivalent to the time saved. Long et al. (1986), 
noted that time-shortened programs were not designed to advance 
the students skills beyond the traditional program, anu simply 
reduce the time required for the individual to complete the 
stated requirements. 
Advance Placement 

Often incorrectly referred to as time-shortened programs, 
advance placement grants students postsecondary credits for 
accomplishments at the secondary level. Although the credits are 
awarded in various ways, each of the advance placement profiles 
has a caveat which distinguishes it from credit transfer. 

In the first profile, individuals who entered the college as 
secondary students in an enrichment program with aspirations for 
postsecondary study, are awarded advance placement. In this 
case, the enrichment program students essentially study certain 
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topics in depth rather than general technical training. As they 
matriculate they are awarded advance placement credits only if 
they enroll in a program to which their enrichment program can be 
applied. 

Breuder and Martin (1985-86), identified a second type of 
advance placement in which college credits are awarded for high 
school courses completed. Long et al. (1986) noted that this 
second type involved college instructors and their high school 
counterparts reviewing the specific course syllabuses or task 
lists. Upon agreement of which high school courses are more or 
less equivalent to introductory level postsecondary courses, 
matriculating students can receive advance placement with the 
written recommendation of their high school instructors for those 
competencies mastered. Breuder and Martin (1985-86) noted that 
in the event a matriculated student demonstrates that the 
competencies for which advance placement credits were awarded, 
have not been mastered, the student would be given independent 
instruction in subsequent courses to develop the deficient 
competencies. 

The third advance placement program profile involves a 
appropriate skills test which allows the student to receive 
advance placement for demonstrated mastery. 

The final advance placement profile was noted by Ierner 
(1987). This profile involves individual students with "C" 
grades who applied for credit transfer in some postsecondary 
institutions. Such students could receive advance placement and 
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not receive the credit transfer until after successfully 
completing a specified number of postsecondary credits at the 
granting institution. 
Career Ladder Articulation Programs 

Each of the next four articulation programs has a career 
ladder. 

One-Plus-One . 

As a career ladder approach, in a One-Plus-One articulation 
program, the receiving postsecondary institution accepts students 
who have completed a one-year diploma (the first one in the one- 
plus-one) program at the receiving institution. common example 
would be a one-year diploma from a proprietary school. Although 
vocational technical diplomas are not awarded at the secondary 
level in Wisconsin, one possible scenario comes to mind. 
Postsecondary articulation programs must address the individual 
who moves into the state of Wisconsin from a state in which 
vocational technical diplomas was awarded at the secondary level. 
Because of the wide range of programs available nationally, broad 
policies should provide for this consideration on a case-by-case 
basis. 

Advanced Skills Programs . 

Long et al . (1986) discusses advance skills and a common 
misnomer. Designed to avoid training duplication, advance skill 
programs streamline fundamentals to make room for the development 
of more advance skills. Such programs emphasize advance content 
aimed at graduating master technicians. The misnomer which is 
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frequently applied to all advance skill programs — as well as some 
time-shortened programs, is the inappropriate use of the term 
two-plus-two which denotes a structured learning sequence 
covering grades eleven through fourteen. 

To avoid the confusion associated with using two-plus-two as 
a collective term, it is best to use the program titles 
identified by Long et al. (1986) for the two main advance skills 
programs: (3> core curriculum for pre-technology) programs , and 
(2) true vocational-technical 2+2 programs in which the entire 
occupational training curriculum begins in grade eleven and 
terminates with the student's completion of grade fourteen. 

According to Parnell (1985) , an advance skill prog:"»a is an 
articulation program which joins the high school curriculum with 
two years of education at a postsecondary institution. If 
properly designed, advance skills programs can provide maximum 
continuity of instruction within and between educational 
institutions. The end product is a highly specialized and 
employable — some may say trained as opposed to educated- 
technician. Warmbrod and Long (1986) argue that the training 
possibilities for advance skills programming are only limited by 
educational resources and* employment trends. 

Given a favorable environment for their development, advance 
skills programs have unparalled advantages. One significant 
advantage deals with separate educational jurisdiction joining 
together for the benefit of students. The focused is on student 
outcomes which were defined in terms of student achievement 

323 



9 

ERJC 



322 



levels and based on employment opportunities. The student 
centered orientation provides the essential ingredients for the 
successful development. 

Core Curriculum (Tech-Prep) Program . 

The main purpose of core curriculum or tech-prep programs is 
to produce better prepared high school graduates in a pre- 
technology curriculum for entry into postsecondary technical 
training programs, core curriculum (Tech-Prep) programs give 
secondary students a broad basic background in technology — a 
strong core of concepts and skills— but do not restrict students 
to making an occupational choice in their junior year. Many such 
programs include agreements that enable matriculating students to 
bypass introductory postsecondary courses and extend the normal 
two-year training program by taking more advanced courses. 
Although the preparation is broader, Long et al. (1986) points 
out that the high school students still receive sufficient 
specific skill training for entry level employment. 

Several examples of an articulated core curriculum program 
are now available. 

1. Oklahoma City's articulation effort, which is built on the 
Principles of Technology, tech-prep curriculum u sloped by 
the Center for Occupational Research and Development (CORD) 
and the Agency for Instructional Technology. The 
articulation responds to community needs for more and better 
trained technicians for high-technology industries in the 
Oklahoma City area, it also is part of a local economic 
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development effort to attract new high-tech industries to 
the city (Long et al., 1986, p. 5). 

2. One leader in this effort is Daniel M. Hull, president of 
the Center for Occupational Research and Development (CORD) 
in Waco, Texas. CORD (1984) has developed a two-year course 
in applied science for junior and senior high school 
students that should improve their knowledge of science and 
math. Several secondary vocational schools throughout the 
nation have adopted this concept and are developing 
excellent course material for this 2-year sequence 
(Lerner,1987, p. 18) 

3. The CORD (1985) Advance Technology Core Curriculum Guide is 
an articulation effort with four post-secondary tracks for 
(a) laser/electro optics, (b) instrument and control, (c) 
robotics and automated systems courses, and fd) micro 
electronics course. 

4. Curriculum 2000, published by the Society of Manufacturing 
Engineers, is an articulation effort involving secondary and 
postsecondary two and four year colleges and articulated 
curriculum development for manufacturing engineer education 
and engineering technology. 

Vocational Technical 2-Plus-2 Program . 

The Vocational Technical 2-Plus-2 Program takes a view which 
is focused on developing advanced skills for a high-technology 
occupational area during grades eleven through fourteen. 

The partnership involves faculty members, administrators, 
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and employer representatives in planning the curriculum and 
deciding what will be taught at each grade level (Warmbrod and 
Long, 1986, p. 29). The curriculum arranges the study of 
mathematics, science, communication, technology, and specific 
technical skills associated with the occupational area under 
study to reach the master technician level of competencies 
through the use of a step-by-step progression terminating at the 
end of grade fourteen. As noted by Warmbrod and Long (1986) and 
Long et al. (1986) , a career ladder approach is built in which 
permits students to exit at the end of grades twelve, thirteen, 
and/or fourteen. 

Long et al.(l986), appropriately points out that the 
vocational technical 2-plus-2 program requires the blending of 
secondary and postsecondary resources. The cess involves the 
development of new, comprehensive, competency-based curricula 
covering the four year period. Success requires the 
establishment of strong, mutually respectful relationships 
between the participating administrators as well as faculty; and 
the development of close, open, collaborative relationship with 
local businesses and industries. The partnership requires a 
substantial investment in planning, development and management of 
formal operational and financing requirements. 
Horizontal Articulation 

Long et al.(1986) specifically identifies one other form of 
articulation which should be mentioned briefly. Horizontal 
articulation facilitates the movement of a student from one 



campus or program to another of the same type. Although this 
types of articulation is currently negotiated at the 
postsecondary level, it can impact directly on 
secondary/postsecondary articulation efforts. 

If the student participating in a vertical articulation 
program can now move horizontally at the end of grade thirteen, 
the additional flexibility can be used as a major selling point 
for initial entry into the vertical articulation program at the 
secondary level. 

Overcoming Obstacles 
The most common concerns about articulation programs 

according to Long et al. (1986), related to staff acceptance, 

institutional turf concerns, poor internal communication and 

inadequate promotion. 

Following an extensive review of literature, Stewart and 

Heiman (1986) concluded that: 

While an increasing number of institutions were found 
to be working on articulation agreements, it appears that 
most reports described individual efforts rather than 
information about establishing articulation agreements. The 
common characteristic found in the reports related to the 
need to establish communication so that duplication of 
efforts could be minimized (p. 115). 

Implementing T nter-Institutional Articulation 
In their study of articulation in vocational agriculture, 

Stewart and Heiman (1986) substantiated that even though 
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secondary and postsecondary vocational programs (agriculture) 
teachers have many mutual perceptions in common, more 
communication should occur. Stewart and Heiman recommend that 
secondary teachers involved in articulation programs should: (a) 
learn more about postsecondary education, (b) visit postsecondary 
programs, (c) teach secondary students about careers requiring 
postsecondary preparation, and (d) refer prospective 
postsecondary students to the appropriate institutions. Stewart 
and Heiman also recommend that post secondary teachers enhance 
the articulation effort by promoting communication with secondary 
teachers. Activities recommended by Stewart and Heiman for 
postsecondary educators to promote articulation include (a) 
appointing secondary teachers to progiam advisory committees, (b) 
visiting individually with secondary teachers at their 
institution, (c) inviting secondary teachers to visit 
postsecondary programs, and (d) developing a policy for awarding 
college credit for documented prior learning. 

In a study of secondary/postsecondary articulation conducted 
for the National Center for Research in Vocational Education, 
Long et al. (1986), also cited in Lerner, (1987) and Warmbrod 
Long, (1986); determined the approaches to articulation -and 
identified common activities as well as barriers to the process. 
They identified two of the general models for articulation 
programs, they were the time-shortened and the advance skill 
mentioned previously. Regardless of the model followed, the 
articulation programs studied had the following characteristics 
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in common: (1) leadership and commitment must be provided from 
the top, (2) faculty (secondary and postsecondary teachers) must 
be involved early, (3) relationships must be based on mutual 
respect and trust, (4) the mutual benefit to all partners must be 
ensured, (5) articulation agreements must bo in writing, (6) 
communication between participants must be open, clear, and 
frequent, (7) initial goals must be modest, (8) responsibilities 
must be clearly defined, (9) curricula must be competency-based, 
(10) the focus must be on mutual goals rather than 
individual/ institutional interest (turf) . 

Lerner (1987) identifies the following twelve steps for 
consideration in implementing an articulation program: 

1. Identify the need for and benefits of articulating with 
other educational institutions in your area. 

2. Identify other educational institutions that would 
benefit from articulating with your school or college. 

3. Meet with the chief executive officers (CEOs) of these 
organizations. 

4. Assign someone the responsibility of directing the 
articulation effort. 

5. Identify the person in the private occupational school 
who can certify transfer students from vocational 
school programs. 

6. Establish clear communication channels within your 
institution and between and among institutions. 

7. Determine the college or university degree program into 
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which the private occupational school students can 
transfer. 

8. Establish whether the transfer will be granted on a 
course-by-course basis or on the blanket concept. 

9. Develop written articulation agreements for execution 
at the institutional level and between program 
departments . 

10. Begin by selecting one or two program areas that appear 
amenable, where faculty members have established 
relationships, and that have a particular need for 
articulation. On^*' these program areas are 
successfully ar^xculated and the benefits made visible, 
use these successes to get other occupational 
departments involved. 

11. Establish a contact person or department at each school 
involved in the agreement. 

12. Provide secretarial support for articulation 
coordinator and faculty to aid their coordination, 
planning, and curricula development. 

13. Establish a system for certifying student competencies 
or educational accomplishments from the articulated 
courses. 

14. Publicize the articulation arrangements and programs to 
students, parents, employers, and community officials 
(p. 18-19). 
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Summary 

Successful articulation programs are focused on improved 
communication among persons at the secondary and postsecondary 
levels, and those policies and practices which facilitate student 
v .ogress. Because of their close relationship to students on 
both levels, teachers are key personnel in the process. Improved 
articulation is a cooperative effort which must involve 
counselors and administrators at both levels. The success and 
extent to which any articulation program is negotiated involves 
credit or recognition for prior experiences. 

Pamell (1985) also cited in Shapiro (1986), offers seven 
specific recommendations for developing cooperative efforts. 

1. All students need a student centered curriculum. The 
barriers to achieving individual excellence for all 
students must be identified and removed. 

2. Unfocused learning will not produce excellence. 
Educational programs must provide the necessary 
structure and substance. 

3. Students must be able to view the total educational 
system as providing a single, coherent program. 

4. Students must see and feel a connectedness between what 
they do and the larger whole between education and the 
rest of the real world. 

5. Students must experience or be able to envision a 
continuity in learning between one institution and 
another. 
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6. Secondary level vocational education curriculum must 
aim at preparing students for broad career areas rather 
than for specific jobs. 

7. Students must see the value in and necessity to develop 
the competencies for continuing their learning 
throughout a lifetime as a means of avoiding 
obsolescence. 

There is no single articulation program which will satisfy 
all situations. Open, student focused communication, is the key 
to successful articulation. If articulation programs are not 
developed between secondary and postsecondary institutions, every 
one loses. Vocational education programs at the secondary level 
may be placed at risk and the postsecondary program will fail to 
develop the necessary feeder programs at the secondary level. 
Ultimately it is the students themselves who have the most to 
lose. Articulation j ograms appear to be one key to success. By 
making post-secondary training programs meaningful, attainable, 
and more attractive to students, articulation can help keep 
future technicians from seeking their initial postsecondary 
training — and employment — outside the local area. 
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Module Development Guidelines 
Introduction 

This module presents the guidelines used by educators 
participating in the High-Technology Training Model Project to 
develop training modules. The training modules developed new 
content for high school and postsecondary courses designed to 
incorporate competencies related to identified new technologies 
being used by local businesses and industries. This module was 
developed as part of the workshop presented to all instructors 
who participated in the High-Technology Training Model Project. 
The workshop was presented in June, 1989 at UW-Stout. 

Purpose 

The purpose of this module is to provide a comprehensive 
description of the guidelines which served as a template for 
educators participating in the High-Technology Training Model 
Project as they developed their training modules. 

Objectives 

This module is designed to: 

1. Describe the training module development guidelines. 

2. Describe the process used to change the competencies related 
to identified new technologies into relevant content for existing 
programs and courses. 



338 



337 



Module Guidelines 
The following guidelines were developed to serve as a template 
for participants as they developed their training module. 
Title 

The title portion of the module contains the name of the 
module. The title is based on the contents of the module being 
developed. It should be succinct and to the point. Other 
content experts should readily recognize the title as part of the 
content . 
Description 

A module course description explains what the module is 
about. It is written by the teacher to clarify the module, place 
the module in the sequence of the course, specify if any 
prerequisites are needed, and suggest the kinds of typical 
activities that students can expect. It should be short and 
concise. If the module is articulated between courses within a 
school, or between the secondary and/or postsecondary level, it 
should be noted here. 
Name and School 

The person (s) who developed the module, and the name of the 
school and address are identified. This will facilitate contact 
if other instructors who might utilize the module have questions. 
Objectives 

Purpose of Objectives . 

Objectives describe in precise terms what the content will 
be and what responsibilities students will have. They usually 
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suggest the nature of evaluation. Objectives provide the road 
for all learners and teachers. They tell you where you're going 
and when you have arrived. Module and specific objectives were 
reviewed for this project. 

1. The module objective states the intended purpose (overall 
outcome) expected of each student after completion of the module 
of instruction. A well stated module objective is usr.ally stated 
in three parts; the condition , performance and criteria 
component . 

2. Specific objectives are the subordinate objectives that 
state what the student must achieve in order to reach the module 
objective. 

Benefits of Objectives , 

Many benefits of objectives may be listed. The following 
are the main benefits discussed in the project: 

1. When students are provided with well -stated objectives , they 
know exactly what will be expected of them. They know what they 
must learn, under what conditions, and how they will be 
evaluated. 

2. If the objective is well written, any instructor can examine 
it and determine what kinds of learning activities might be 
appropriate for learning the objective. 

3. A well-written objective also reveals the major learning 
materials and resources (tools, equipment, supplies, etc.) needed 
for learning the content. 

4. Well-stated objectives tend to help everyone involved 
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(teacher, evaluators, aids, media specialist, etc.) in the 
teaching-learning process to understand what the student is 
attempting to do. 

5. By informing learners of the objectives of the session, 
learning is enhanced. 

6. Precision in the instructional process is enhanced if the 
objective is stated. Well-stated objectives remove the ambiguity 
in the learning process. By stating the conditions under which 
the learner must perform, exactly what performance is required, 
and the specific criteria for evaluation, the fuzziness out of 
both the teaching and the learning process is reduced. 

Components of Objectives . 
1. The Condition Component: 

a. The condition component of the objective describes the 
setting in which the student will be required to perform the task 
to demonstrate learning. The condition is often referred to as 
the given and should be written just as carefully for cognitive, 
psychomotor, or effective objectives. The Figure 1 provides 
examples of the condition component. 



Figure 1. Condition Components of the Objective 
1 . Things 

a. Given specific tools 

b. Given materials or consumable supplies 

c. Using test instruments, manuals, specifications, 
etc * (Figure cont.) 
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d. Given a drill, sterilizer, or other specific piece 
of equipment 

e. Provided with mock-ups, devices, etc. 

f. Given objects encountered on the job - soil, 
broken belt, patron, recipe, etc. 

2 . Situations 

a. Using actual work 

b. Under some specified situation 

c. Presented with picture, problems, case study 

d. Given a work order, verbal instructions, 
blueprint, etc. 

e. Provided with results of a diagnostic test 

f. Provided with the data, measurements, parameters, 
map, schematics 

g. Given lists of terms, parts, tools, etc. 

h. Given a field situation 

i. Given numbers, figures, or problems 

3 . Restrictions 

a. Without the use of texts, books, manuals or 

references 

b. Without any aid 

c. Without specific tools and equipment such as 
calculator, special tools, tables, charts, etc. 

b. The following tips might be helpful in writing the 
condition component of objectives: 
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(1) Avoid a long list of specific tools and equipment. 
This generally has little use and prevents the objective from 
becoming so long. 

(2) Do not include any objects, situations, or 
restrictions that are jbvious to all concerned, such as "given a 
laboratory, welding gloves and tanks." Do mention any special 
restrictions under which the students will have to perform. 

(3) Do not include anything in the condition that the 
learner should not have during the testing situation. If the 
student is required to determine what materials are needed before 
performing a task, do not list those materials in the condition. 

(4) Avoid indicating reference to how the student will 
learn the task. Do not use phrases such as "given a lecture on," 
"given the required reading materials," and so on. The condition 
focuses on the evaluation/testing situation - not the learning 
situation. 

(5) Do not make the condition too specific. For 
example, "given two pieces of wood, 4 inches by 4 inches," might 
be too restrictive. "Given two pieces of material," would 
probably be sufficient. 

(6) The condition should resemble the condition under 
which the student must perform the task. If a worker must be 
able to write up a purchase order on the job from verbal orders, 
then "given verbal orders ..." should be the condition 
specified in the objective. 
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2. The Performance Component: 

a. The performance component is nothing more than the task 
or competency statement. However, the task statement may be 
expanded, additional qualifier added, or altered to describe the 
performance accepted in the education or training setting. 
Figure 2. provides examples of the performance components. Note 
that the task statement has been expanded or made more specific. 



Figure 2. Performance Component of the Objective 

Performance Component of 
Tasks Objective for that Task 

1. Lay out a rafter lay rafter to span 

12 feet 

2. Clean, gap, and test remove, clean, and 
spark plugs gap the plugs and 

replace in engine 

3. Prepare purchase order take verbal order, 

type, and check 
account code 

4. Install transmission install and perform 

checks 



b. Consider the following when dealing with the 
perfori.vmce component. 

(1) Make sure it includes the task or competency 

statement . 

(2) Have another teacher describe exactly what the 
student should be able to do to demonstrate mastery of the task, 
If they can, you have the performance clear. 

(3) The performance component should describe the 
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final (terminal) testing situation for the student to be 
considered competent. 

(4) Keep the instructor performance separate from 
student performance. Never use "will teach," "will demonstrate," 
etc., or other instructor behaviors. We are interested in 
student performance. 
3. The Criteria Component: 

a. The criteria component focuses on how the learner will 
perform the task for the teacher to conclude that the task has 
been accomplished (job entry level). Students can perform almost 
all criteria of a task except perhaps one — speed. Figure 3 
provides examples of product and process criteria. 

b. The following tips may be helpful in writing the 
criteria component of the objective: 

(1) Keep the criteria at a level to ensure entry-level 
job employment by the student or pre-requisite for the next 
objective. 

(2) Reduce the criteria dealing with speed as would be 
required for an experienced worker performing the task on the 
job. 

(3) Do not connect the criteria to instruction. Do 
not use phrases such as "following criteria in textbook" or 
"according to specs in handout." These resources will change - 
criteria should not. 

(4) Speed may be important on the job. If it is 
mentioned, a reasonable time limit in the criteria (or include 
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time as an item on the performance test) . 



Figure 3. Product and Process Criteria 



Product Criteria 



(The finished product) 



Process Criteria 

(How the student 
performs the task) 



* 
* 

* 
* 
* 

* 
* 
* 



Examples 

+0.004 inch, +1 mm, +3 degrees 

According to manufacturer's 
specs 

Within 8% of actual reading 
Within 99% accuracy 
With no errors 

Must agree with instructor's 
measurements 

To customer's satisfaction 
Following manufacturer's 
Within 20 minutes 



Not exceeding flat-rate time 
by more than 30% 

Performing all steps in 
sequence 

Following safety practices 



Content Outline 

The content outline is usually the beginning step in the 
development of a module, it is a skeleton framework of the 
module content developed by the teacher. It is similar to a 
topic outline but dealing with the content to be covered in the 



346 

345 



module. The teacher may use a variety of resources to develop 
the module, such as another outline as a base, a personal 
analysis or existing materials or guides. A third level content 
outline is usually sufficient to determine the main concepts in 
the module are being covered. 
Methodology 

This section provides examples of teacher actions to 
accomplish the objectives of the module. How the teacher will 
facilitate the activity (ies) to accomplish the objectives listed 
in this section. 

EXAMPLES 1 . 

1. Obtain additional materials 

2. Make transparencies 

3. Write worksheet 

4. Provide students with objective sheet 

Additional suggested activities should be listed as needed. 
EXAMPLES 1. Relationship to Objective . 

The activities identified in the module should be directly 
related to the objective or sub-objective of the module. In 
fact, as stated in the objective section, a well-written 
objective will also include what the student will do to learn the 
material . 

EXAMPLE 2. Be Specific to the Student . 

The activities identified in the module should also include 
the kinds of things that students will be engaged in to learn the 
content. Will students be involved in: 
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1. Demonstrating something to other students 

2 . Identifying parts 

3. Testing a circuit developed 

4. Writing a report 

5. Programming a routine 

6. Recalling information 
Resources 

Resources are references needed by students and or teacher 
to accomplish the objectives of the module. They are identified 
in this section. Someone not familiar with the content can 
easily determine the special materials needed to teach the 
module. Resources may include: 



1. 


Books 


2. 


Pamphlets 


3. 


Films 


4. 


Brochures 


5. 


Maps 


6. 


Materials 


7. 


Equipment 


8. 


Supplies, etc. 



Evaluation 

The evaluation used must be related to the module, it 
should be based on the objective and the performance standard 
noted in the objective. The objective may call for cognitive 
information or a skill. 

Paper and pencil test is usually used to measure the 
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knowledge of the student. This test may take the form of an 
essay, multiple choice, true false, etc., test to measure the 
students knowledge. 

Process/ Product evaluation is basically a performance test. 
In process evaluation, the teacher evaluates the student as they 
complete the activity. In product evaluation, the completed 
project is evaluated using pre-determined criteria. In any case, 
the evaluation criteria should be identified and listed in this 
section. Any exams, product or process checklists developed 
should be included. Remember: These modules will be shared with 
various teachers in the field. 
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Evaluation 
Intr oduction 

Proper evaluation in a major key to the success of any 
education or training program. As learners, students and 
trainees iaust be given feedback as to their progress, and 
educator* and trainers roust be able to properly judge the 
learner's preparedness to begin a task or job. 

Evaluation of the laarner may be very simple or very complex 
depending on tb<* &**ch of the program and the level the learner 
is at. Xt is ijaperacivo that the educators and trainers knows 
how aaix h the type of evaluation to the program to be taught. 
Mutatio nal Procte r.; Ev aluation 

Evaluation is an essential element for the improvement and 
strength of. any program and course. Learner, educator/trainer, 
and management feedback are all important factors in building a 
strong and sound program. 

Purpose 

This module is designed to provide the information needed to 
write complete measurable objectives, match the learner 
evaluation techniques to objectives, and selection of evaluation 
methods best suited for a given program. 

Object ives 

The objectives for this module are: 

1. Write a clear measurable objective. 

2. ' Write an evaluation scheme that properly matches the 
objective. 
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3. Evaluate learning situation using a variety of methods. 

Syste ms Approach to Evaluation 
Measura b le Objective s 

Good, c3ear, measurable objectives which are understood by 
educators/trainers and alike, form the foundation for a 
successful evaluation plan. 

Measurable Objectives must have; 

1. Condition - Under what conditions is the task to be done? 

2. Intent - Exactly what is the task the student/trainee is to 
do? Usually described in terms of an action verb. 

3. Level of performance - What is the standard or to what 
degree must the task be performed? 

Appendix A contains a model depicting the elements of 
measurable objectives. For more information on objectives and 
the domains and taxonomies of learning, see Gronlund (1985), 
Stating Objectives for Classroom Instruction 

Measurable objectives are based on the knowledge, skills and 
competencies required to perform a task, solve a problem or make 
a decision. 
Evaluating Learners 

Various Ways to Evaluate Student's Performance . 
1. Tests 

a . oral 

b. written 

(1) essay 

(2) objective 
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(a) true and false 

(b) multiple choice 

(c) matching 

(d) completion 

(e) listing 

(f) identification 

- drawings 

- actual object 

2. Observation instruments or techniques 

a. progress charts 

b. rating scales 

c. checklists 

d. interviews 

3. Gronlund (1985) and Green (1975) provide good examples of 
these tests and instructions for their development. 

Matching Evaluation Methods to Performance . 

One must remember to select the type of evaluation based on 
what the objective requires. Two examples follow. 
Example 1: 

Objective: Without the aid of reference material, relate 
how the first amendment affects our lives today. 

Poor Evaluation: Write the first amendment. 

Good Evaluation: State the first amendment, then write 
three short essay examples of how the amendment is being used or 
abused today. 
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Example 2; 

Objective: Given a malfunctioning radio, the student will 
be able to repair the radio to proper working condition. 

Poor evaluation: List five items that may cause a radio to 
malfunction. 

Good Evaluation: Give the student a malfunctioning radio 
and have them repair it. The student will be evaluated based on 
if the radio works properly or not. Also, factors such as the 
time required to fix the radio, could be included in the 
evaluation. 

Masterv/Competency Based Evaluation . 

Mastery/Competency based evaluation evaluates the 
individual's performance as compared against the acceptable 
minimum criterion. The following example is provided. 
Example: 

Objective: Given a PC and typical business documents, the 
student will be able to key enter 70 wpm with three or fewer 
errors. 

Valid Evaluation: Give the students a typical business 
document, a PC, and have them enter text for 10 minutes. Print a 
copy of text entered and calculate the wpm and count the number 
of errors per 70 words. 

For more information on learner evaluation refer to Mager 
(1984), and Green (1975). For more information on 
mastery/competency based evaluation, refer to Blank (1982) . 
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Evaluating Learning Programs 

Three Ways To Rate Training Success . 

1. Contribution to goals - How much has the education/training 
helped with the organizations performance problems? 

Example: Company "B" produced 100 widgets per hour. After 
training, company "B" produced 150 widgets per hour. Training 
contributed 50 widgets per hour. 

2. Achievement of learning objectives - This type of evaluation 
requires that each learner be tested on each learning objective 

isted for the program. 

The evaluation include post-training measurement of actual 
on-the-job use of the new behaviors or the direct application in 
an appropriate academic environment. 



Obiective 


1 


2 


3 


4 


5 


6 


7 




1 


X 


X 


X 


X 


X 


X 


X 




2 


X 




X 


X 




X 


X 


36 of 4f< 


3 


X 


X 




X 


X 


X 




objectives 


4 






X 


X 


X 




X 


were met. 


5* 


X 










X 






6 




X 


X 


X 


X 


X 


X 





Note:* one should note that only 2 out of 7 students 
achieved objective #5. The educator/ trainer may want to look 
into this objective more carefully. 
3. Evaluation by perceptions 

a. Ask opinions of students on: 
(1) content 
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(2) instruction 

(3) facilities 

( 4 ) interest 

(5) appropriateness 

b. Ask open ended positive/ negative questions 

c. Likert scales 

(For information on statistical interpretation of 
likert scales, reference any statistics book.) 

Examples: High-Technology Training Model Project, Training 
Module 

- teacher's module evaluation 

- student's module evaluation 

- module development evaluation 

Copies of these forms are included in Appendix B, as 
examp- 

Evaluation System for Industry Based Training Programs 
Introduction 

Program and trainee evaluations are also important for 
industry based training programs. The evaluation principles 
discussed in the first part of this module are equally applicable 
to the designs of evaluations systems for industry based training 
programs. Valid evaluation techniques are essential. To achieve 
this one must have specific objectives and appropriate evaluation 
techniques. 

There are some differences, however. Often adults are 
afraid of tests and do not want to be graded on an A, B, C, etc. 
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basis. This is especially true if they have not been "in school" 
for awhile. Also, written tests are not always the most 
appropriate measuring technique. 

Application of a systems program is usually very helpful. 
The systems model directs the trainer to the most critical 
variables to measure and helps to identify relationships between 
inputs, processes and outputs. 

Designing Evaluation to Improve Training Programs 

Evaluation is a two-edged sword. The information provided 
from an evaluation can generate feedback for learners that 
reinforces their work, help instructors improve their training 
program, and give managers data that help them determine the 
value of the training program. 

Evaluation can also have negative effects. It can cause 
trainees to become anxious and lose interest in training 
activities. Instructors can become wary of the outcomes of the 
evaluation and the potential impacts on their work. Supervisors 
and managers can become concerned that an evaluation will 
undermine a training program they feel is important. 

An effectively designed evaluation can draw upon the strong 
points just mentioned and avoid the negative aspects. 
Participants in training programs need feedback to provide 
reinforcement and help them determine the outcomes of their work 
in the training activities. In like manner, trainers need to 
know the impacts of their training activities and have 
information that will help them improve their training programs. 
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Supervisors and managers need to have feedback on the impacts of 
the training programs they have sponsored. 

The first part of Appendix C describes an evaluation system 
that was used in a series of twelve training programs provided 
for a large corporation. The evaluation proved to be effective 
in providing information that was useful to the trainees, helped 
the instructors improve the workshop design and content, and 
assisted managers in assessing the value of the training program. 

The second part of Appendix C presents more general concepts 
concerning the design of evaluation systems for training 
programs. Also, the evaluation described in the first part is 
expanded to include a follow-up phase. 
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Measurable Objective's 



refill rHtiftnQ nf* PpT*fnT*m^TipP 


Trmtrnpti final Tntpnt 




States under what conditions the 


What we expect the student to be 


Provides a minimum standard of 


student will be able to perform. 


able to do. 


achievement. 


• Given a problem of the following 


• To write 


• Identify all... 


class... 








• To recite 


• State three... 


• Oivpn a lict of 








• To differentiate 


• Cut within ± of 1/4" 


• Given any reference of the 






learner's choice... 


• To solve 


• Remove 60% of... 


• Given a matrix of 


• To identify 




intercorrelations. . . 








• To construct 




• Given a properly functioning. . . 


* 






• To list 




• Without, t.hf* aifi of reference 

IT ItlivUU bUv (UU Ul ivivi vllvviii 








1 To compare 




• Without the aid of slide rule. . . 






I 


• To contrast 





At the completion of this lesson the student will be able to: 

• Given the Paxton sample of the 35 most common woods, the student will be able to identify all the woods in the group of 
samples. 

• Given the carburetor specification of Ford k >cor Company, the student will be able to state the two different jet sized found in 
carburetors made by the Ford Motor Company. 

n f* i Dr. Richard Gebhart 

° UW-Stout 



APPENDIX B 
Training Module Evaluation Forms 

- Teacher's Module Evaluation 

- Student's Module Evaluation 

- Module Development Evaluation 
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TEACHER EVALUATION 
II lull -TECH MODULE 



8/23/fcj 



TEACHER 



DATE 



HJLE TITLI- 
OATE(s) USED 



CLASS (cs) 



TIME REQUIRED TO COMPLETE THE MODULE (MINUTES ) 

Directions: Please list the information requested above. Then read each scatemeuc 

below and respond based on your experiences with the module. Use the 

following responses. 

1 " SD ■ Stroncly Disagree 

2 = D ■ Disagree 

3 = U = Undecided 

4 = A = Agree 

5 = SA ■ Strongly Agree 

If you have any comments, write them by the statements or at the end of the form.. 



EVALUATION STATEMENTS 



SD 
I 



Responses /Comm ents 



D 

2 



U 

3 



A 
4 



SA 
5 



MODULE 



4. 



6. 

7 

8. 



10. 



1. 


The Module 






3 


4 


5 




The Module 






3 


4 


5 


3. 


Objectives 






3 


4 


5 



Activities in the Module were effective in 
stimulating learning 



Audio visual aids and/or resource materials 
were effective * 



I had the classroom and lab equipment 
required to effectively use this Module. 



Evaluation procedures were effective. 

This Module introduced new content, 
concepts , skills, etc. in my class... 



The knowledge *s;id/or skills students 
learned are relevant to the needs and 
trends in industry 



This Module fit well with the rest 
of my course content „ 
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High-Tech (10/10) 



STUDENT EVALUATION FORM 



SCHOOL 
CLASS 



DATE 



MODULE 



Directions: 



Read the statements on the left. Use your experiences in the 
unit just completed in class to answer each statement. Use the 
following responses. There is space for comments at the end. 

1 = SD = r ongly Disagree 

2 = D = Disagree 
-3 « U - Undecided 

4 = A = Agree 

5 = SA = Strongly Agree 



EVALUATION STATEMENTS SD 
1_ 

1. I liked this learning unit.. 1 

; I learned some useful things 1 

3. The learning activities were interesting 1 

4. The activities, tasks, and/or assignments 

were helpful 1 

5- I learned some things that will be useful 

in the future 1 

6. I learned some new things 1 



Responses 
D U A SA 
2 3 4 5 



2 
2 



2 
7 



3 
3 
3 



3 
3 



4 
4 
4 



4 
4 



5 
5 
5 



5 
5 



7. What was the most important thing you learned? 



8. Hov could this unit be improved? 
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On/) 



h i -l F.i.ii trailing project 

MOnill.K KVAUIATWN KORM 



M 



E XAMU 



Scale: I * Needs Improvement 
2 « Satisfactory 
:i « Verv Good 



Dimension 



Rating 



Comments 



1. TITLE: 

The module contains a name 1 2 3 

2. DESCRIPTION: 

Information is provided as to where the 

module fits in a sequence of a unit/ task/ 

course 1 - ° 

3. OBJECTIVES: 

A. Module objective is stated 1 2 3 

B. Specific objectives are listed i 2 3 

4. CONTEST OUTLINE: 

A first, second and third order outline 

of the content is presented 1 2 3 

i 

5. ACTIVITIES: 

Are identified which will help students 

learn the content 1 2 3 

6 . RESOURCES : 

A variety is listed which will help 
students/teachers accomplish objective- ..-123 

7. EVALUATION: 

A. Evaluation criteria identified. .... 1 2 3 

B. Evaluation procedures given 1 2 3 



ADDITIONAL COMMENTS: 
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Desi gning Evaluations to Improve Training Programs 
Evaluation System 

The following evaluation system was developed to evaluate a 
statistics and process control training program. The program was 
forty hours in length and conducted during a two-week time 
period, trainees attended morning or afternoon sessions. 
Training was conducted on-site. 

The goal of the company was to develop an effective quality 
program throughout all departments. Thus, a cross-section of 
employees in a department or plant site was enrolled in each 
training program. As a result, participants had a wide range of 
educational levels, statistical expertise, and technical 
knowledge. Ages ranged from these in their early 20' s to people 
near retirement. 
Evaluation Design 

A pre and post-test design was used in order to identify 
gains in knowledge during the training program. A cognitive test 
was developed for the training program and was used as the pre 
and post-test. Copies of the test were sent to the on-site 
coordinator for the training program. This person distributed 
the tests to the prospective trainees and collected and returned 
the completed tests to the trainers two weeks prior to the start 
of the training program. This allowed the pre-test to be 
analyzed and the instructors to review the results before 
initiating the training. Adjustments in the training program 
content and pace of instruction could be made based on the test 
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results . 

In order to reduce test anxiety, trainees were not asked to 
place their names on the pre or post-test. Instead, a 3 X 5 card 
with a number on it was clipped to each of the pre-tests. The 
same number was recorded on the test booklet. Trainees were 
asked to keep the card and place this number on the post-test 
when it was given at the end of the training program. A few 
trainees forgot to retain their numbers; however, overall the 
process worked quite well, there was no indication of test 
anxiety or concern over how the test results might be used by the 
trainers or supervisors. 

During the last session of the training program, the post- 
test was administered. This was the same test as used in the 
pre-test. An alternative form of the test probably would have 
been more appropriate in a measurement sense. However, the time 
and economic considerations, and activities required to develop 
and certify a parallel form of the pre-test made it impractical 
to do this. The questions on the pre-test may have sensitized 
the participants to some topics, however the participants did not 
comment on or ask about the questions on the pre-test during the 
training programs. 

After the post-test had been administered, the participants 
were also asked to complete an evaluation form on their 
experiences in the training program and their perceptions of the 
usefulness of what they had learned, participants did not have 
to place their names on this survey form. They were also 
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encouraged to make suggestions for how the training program could 
be improved. 

Test Validity . 

In order to assure that the cognitive test contained 
appropriate or valid items, the training program objectives were 
analyzed to identify the types of competencies participants would 
need to have in order to meet these objectives. For example, one 
of the objectives was that the participants would develop a 
knowledge of basic statistics to include proportion, range, mean, 
standard deviation, median and mode. This objective was analyzed 
and the following knowledge levels were determined. 

LEVEL SAMPLE COMPETENCIES 

Knowledge: Identify the symbols and list the formulas 



After this analysis had been done, it was a relatively 
simple task to write test items to measure knowledge or 
understanding of the topics related to each objective. A 
multiple choice test item format was used to simplify and speed 
up test scoring, Item analyses were regularly run on each of the 
tests, these item analyses were reviewed to determine any items 
or responses which needed to be revised. One or two revisions 
were made after the first training program. In addition, these 
reviews identified typical errors trainees made on the test 
questions. 



for the basic statistics. 



Application: 



Given a set of data, calculate and interpret 



the appropriate statistics. 
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The second instrument developed to evaluate the training 
program was a rating scale. A systems analysis was done on the 
training program to identify components which should be evaluated 
in the rating scale. A majority of the items in the rating scale 
dealt with the processes and resources used in the training 
program. For example, participants were asked to evaluate 
whether the instructors were able to effectively answer their 
questions and if the training materials were useful. A few of 
the items were concerned with inputs and outputs. For example, 
participants were asked to rate the statement "I learned a lot in 
this training program." A copy of this rating scale is included 
as Appendix D to this module. 

Selected R esults . 

More detailed results for one of the training programs are 
given in this section. A comparison or pre and post-t «3t results 
for several of the training programs will also be given at the 
end of this section, these results are being presented to give 
an idea of the types of analyses that can be used and the 
information available from the evaluation of a training program. 

Before looking at the pre and post-test comparisons, I want 
to mention that the pre-test item analysis for each group was 
reviewed to identify the participants' strengths and weaknesses. 
The item analysis presents a summary of the participants' 
responses to each item in the test. Therefore, it is possible to 
determine concepts that are known by almost all of the 
participants and those that only a few participants know. This 
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information was used by the instructors to adjust the content and 
pace of instruction. 

After each training program had been completed and the post- 
tests scored, the post ani pre-test scores were compared to 
identify gain scores. Only the participants with both pre and 
post-test scores were included in this analysis. This 
restriction could create some potential biases. More detailed 
analysis were done to determine potential impacts of this 
restriction. It was found that some of the individuals who took 
the pre-test were not able to participate in the training 
program. In some instances, participants forgot their number and 
were unable to place it on the post-test. A few participants 
missed the last session of the training program. And, a few 
participants did not take the pre-test. An analysis of the 
average pre-test scores for those who did not take the post-test 
revealed that this average was not significantly different from 
the pre-test scores of those who had also taken the post-test. A 
similar analysis of the average post-test scores for those who 
did not have pre-test scores revealed that there was no 
significant difference when contrasted with the average score for 
those who had taken both pre and post-tests. Based on these 
analyses, the author concluded that if any biases were present, 
they were small in size. 

Table 1 presents the pre-test, post-test and gain scores for 
the participants in one of the training programs. Gain scores 
are determined by subtracting an individual's pre-test score from 
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his/her post-test score. A minus score indicates a post-test 
score that is lower than the pre-test score. (See trainee 34) 
The average gain score for this group was 8.18 points, this may 
not appear to be a large change; however, it was a 39 percent 
increase in performance and was statistically significant, in 
other words, it was very unlikely this difference could have 
occurred by chance. Also, the test items were based on a sample 
of the competencies gained and these did not enumerate all of the 
new knowledge acquired. There were forty items on the test. 

A correlational comparison of pre and post-test scores was 
also run for each training program. An example of the scatter 
diagram for the data in Table 1 is given in Figure 1. This 
analysis indicates that those who have high pre-test scores also 
tend to have high post-test scores. This is a logical outcome 
for an effective training program. Those who enter with more 
knowledge should be able to maintain this edge. 

A comparison of the participants' gain scores with pre-test 
scores is given in Figure 2. This analysis indicates that 
individuals with the lower pre-test scores tended to have the 
lar-er gain scores. Again this would appear to be a logical 
outcome from an effective training program. First, the fact that 
those who had low pre-test scores, or less knowledge, were able 
to gain substantial amount of knowledge is a positive outcome and 
would indicate that the training program did not go er their 
heads. On the other end of the spectrum, those with high scores 
also gained knowledge. It must be remembered that f»*r aid not 
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have as much space on the measuring instrument to show gains as 
those with low pre-test scores. In essence, they are capped from 
showing substantial gain scores. 

A summary of the participants' evaluation of the training 
program is given in Table 2. The moan, standard deviation and 
percentage of participants that agreed with each statement are 
given in the table. The average response represents the typical 
feeling of the group. Variations in reactions, opinions or 
feelings are represented by the 
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TABLE 1 



Pre-Test and Post-Test Results Statistics and Process Control 
Training Program 







Test 


Results 




Trainee 


# 


Post 


Pre 


Gain Scores 


2 




30 




26 


4 


3 




31 




23 


8 


4 




28 




15 


13 


5 




26 




17 


9 


6 




36 




29 


1 


7 




35 




23 


12 


9 




26 




21 


5 


10 




36 




25 


11 


11 




2u 




8 


12 


12 




25 




16 


9 


13 




26 




21 


5 


14 




25 




13 


12 


17 




22 




16 


6 


19 




33 




27 


6 


22 




29 




16 


13 


24 




27 




23 


4 


25 




30 




26 


4 


26 




27 




19 


8 


28 




25 




18 


7 


29 




33 




16 


17 


31 




27 




19 


8 


34 




17 




20 


— 3 


36 




22 




21 


1 


37 




29 




22 


1 


38 




30 




11 


19 


39 




40 




25 


15 


40 




31 




22 


9 


41 




27 




18 


9 


'tZ 




34 




29 


D 


44 




32 




26 


6 


45 




33 




23 


10 


46 




?5 




19 


6 


47 




43 




37 


6 




X 


29 


.09 


20.91 


8.18 




s 


5.52 


5.76 


4.45 




N 


33 








#Scores 


for persons who completed the 


pre-test 


and post-test. 



o t test of differences: t * 10.568 

p < .001 

* Conclusion - There was a significant gain in knowledge. 
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Figure 1: 

Comparison of Pre and Post-Test Results 
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Figure 2: 

Comparison of Pre-Test and Gain Scores 
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TABLE 2 

Participants' Evaluation of Training Program 



Characteristics of the 'raining Program 

N=33 



Results 
S 



% Agree 



1. This program was worthwhile .... 

2. I enjoyed using MINITAB 

3. MINITAB can be helpful in my work . 

4. I gained a better understand of 
statistics 

5. I will be able to use the contents 
of this training program 

6. I liked the opportunity to work 
with my own data 

7. There should be more opportunity to 
apply the contents to practical 
problems related to my work .... 

8. I knew most of the content of this 
training program before wg 
started the sessions 

9. I learned a lot 

10. The instructors' explanations were 
clear 

11. The instructors effectively .... 

12. The instructors were able to relate 
the content of this training 
program to practical problems . . . 

13. The instructors were well prepared. 

14. I plan to use some of the techniques 
presented 

15. Too many topics were included in 
the program 



3.9# 


.8 


77@ 


4.0 


.9 


77 


3.5 


1.1 


47 


4.2 


.9 


86 


3.5 


.9 


50++ 


4.1 


.7 


84 


3.5 


. 9 


46 


1.7 


.9 


7 


3.8 


.9 


64 


4.2 


.6 


93 


4.1 


.7 


82 


4.1 


.8 


87 


4.3 


.7 


59 


3.7 


.7 


59 


3.0 


1.1 


27 
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TABLE 2 (Continued) 



Results 

Characteristics of the Training Program 

N=33 X S % Agree 

16. Too few topics were included in 

the program 2.6 1.1 14 

17. The visuals used were helpful ... 4.1 .9 86 

18. The handouts were helpful T . . . . 4.3 .7 95 

# KEY 

1 = SD = Strongly Disagree 4 = A = Agree 

2 = D = Disagree 5 = SA = Strongly Agree 

3 = U = Undecided 

@ percent of respondents who agreed or strongly agreed with the 
statement . 

+-v 11 percent disagreed with this statement, 
standard deviation. The larger this number the greater the 
variation in responses given by the participants. When this 
number is 0 it indicates that all participants have selected the 
same response. The percent agreed (% agreed) column is a 
combination of the percentage of participants that marked the 
"agree" or "strongly agree" response on the rating scale. For 
oxample, 77 percent of the trainees selected the "agree" or 
"strongly agree" response for item 1, "This program was 
worthwhile. " 

A review of Table 2 indicates that the training program 
content, materials and instructors received consistently high 
ratings. The pace of instruction appeared to be fairly 
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appropriate. (Note items 15 and 16.) 

Participants reported that they enjoyed using the MINITAB 
Software (Item 2), gained a better understanding of statistics 
(Item 4), and planned to use some of the techniques presented 
(Item 14) . 

Many participants also indicated an interest in more 
opportunities to work with their own data (Item 7) . m each 
training program, participants were encouraged to bring in 
samples of the data with which they worked. These were woven 
into problems during the training workshop. However, there were 
limitations on the amount of time that could be given to working 
with their data versus developing new concepts. 

The training program instructors found the rating scale feed 
back to be very helpful. It provided a concise summary of the 
participants' feelings and opinions related to their experiences. 
Their responses to the items and written comments at the end of 
the survey also provided information that was useful in modifying 
the training jgram. 

Perceptions Versus Test BfiSillfcs . 

Out of curiosity, the author did a special analysis 
contrasting participants' opinions of the amount of knowledge 
they had gained during the training program with the actual test 
outcomes. This was done in a selected training program and was 
not completed for each of the twelve evaluations, in this 
training program, participants were asked to place their id 
numbers on the opinionnaire as well as the test. A visual 
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Figure 3: 



Telationship Between Gain Scores and Trainees' Perceptions 
of the Amount They Lear ned 
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comparison of the relationship between perceptions and test 
outcomes is given in Figure 3. The correlation between the 
perceptions of the amount learned and gain scores was .25. This 
correlation would have been higher if those who perceived that 
they had "learned a lot" had consistently high gain scores. 
However, several of the trainees who indicated they "learned a 
lot" had moderate to low gain scores. 

It would appear that in situations where the participants do 
not need to identify themselves and are free to respond based on 
their perceptions, that they can give a moderately accurate 
indication of the amount they have learned. It is very unlikely 
that this relationship would hold true if there was pressure for 
grades, merit ratings, or certification. 

Comparison Groups . 

Because of the nature of the training program and the 
settings in which the training programs took place, it was not 
possible to identify a control or comparison group. In each 
setting, most of the staff of a department or plant site were 
involved with the training program. A quasi experimental design 
could have been used with a comparison group selected from 
another department or plant site. However, logistics and cost 
precluded the use of this option. 

The host company's decision to provide this training program 
at a variety of plant sites made it possible to use a modified 
time series evaluation design, in this design, outcomes are 
contrasted over a period of time to determine if there are 
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consistent patterns, if the training program is effective, one 
would expect that the changes between pre and post-test scores 
would be similar in all groups. In other words, and effective 
training program would have significant gain scores in most, if 
not all, training sites. This was true in the twelve settings in 
which this training program was offered. 

It should be noted that in this type of analysis that it 
would be possible for a pre-test average score in one group to be 
higher than the post-test score in another group. This could 
occur when one group has had very little exposure to statistics 
and nrocess control while another group has had extensive prior 
experience with these areas. There were differences in average 
pre-t- .t scores from setting to setting as depicted in Table 3 
which presents the test results from several of the training 
programs . 

If necessary, an analysis of covariance could be used to 
statistically equate the pre-test scores for the groups. The 
type of statistics used will depend upon the variations in the 
groups involved and the amount of statistical sophistication 
desired by the decision makers involved with the training 
program, in many instances, a visual review of the gain scores 
for each of the training programs will be enough to indicate 
whether a program is effective. 
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TABLE 3 

Test Results From Eleven Training programs 



Training Program Average Test Results 

Pre Post Gain t Decision 

Value 



A 


27.9 


34.2 


6.3 


9.87 


SIG 


GAIN 


B 


20.5 


26.8 


6.3 


7.48 


SIG 


GAIN 




Zo . 4 


35.2 


6 . 8 


8 . 78 


SIG 


GAIN 


D 


21.3 


29.6 


8.3 


7.24 


SIG 


GAIN 


E 


26.2 


33.0 


6.8 


5.9 


SIG 


GAIN 


F 


27.6 


35.4 


7.8 


9.61 


SIG 


GAIN 


G 


26.1 


32.3 


6.2 


7.05 


SIG 


GAIN 


H 


20.9 


29.1 


8.2 


10.06 


SIG 


GAIN 


I 


20.7 


27.8 


7.1 


7.26 


SIG 


GAIN 


J 


19.9 


26.3 


6.4 


7.67 


SIG 


GAIN 


K 


22.8 


29.1 


6.3 


13.3 


SIG 


GAIN 



#Decisions are based on the results of the t test of differences. 
All t tests were significant at the .01 level. 

Summary 

This evaluation system was designed for a specific training 
program delivered on-site in a variety of locations throughout 
the United States. It was planned to provide information that 
could be used by the trainers, participants and managers. Major 
goals of the evaluation were t< identify how effective y the 
training program was meeting its objectives, provide feedback for 
the trainers and to generate information that could be used by 
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management in determining the value of the program. 

From the perspective of the trainers, the evaluation system 
was successful. Information from the test and the rating scale 
was very useful in refining the design of the training program 
and communicating its impacts to management. Participants did 
not indicate any negative reactions to the evaluation process and 
many asked to receive their test scores. 
General Design Principles 

This section presents major factors that should be 
considered in evaluating a training program. These would apply 
to the evaluation of most training programs. 

Relevant and Useful Data . 

The evaluation must produce data that are of value to 
trainers, trainees and managers. Trainers need to know the 
trainees' entering level of competency, progress in the program 
and exit level of competency. They will also want to know the 
trainees' reaction to the program. Trainees are interested in 
their progress and the degree to which they have met the 
objectives of the training program. Managers will want to know 
what competencies have been acquired, areas in which more 
training is needed, and their staff members' reactions to tha 
training program. 

The starting point for the design <»f the evaluation is a 
valid set of training objectives. These objectives will clearly 
define the exit competencies of participants in the training 
program. They should be based on a needs assessment that 
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identifies and prioritized the competencies required to improve 
quality and performance. The evaluation must determine the 
degree to which these objectives have been attained. 

Information needs of the trainers, trainees and managers 
should be kept in mind as the evaluation is designed, it may be 
necessary to discuss these needs with representatives of each 
group. As the evaluation is designed, the evaluator must keep in 
mind the following two questions: (l) what information is 
needed?, and (2) When is it needed? 

Design To Fit The Situation . 

During the design process, consideration must be given to a 
variety of situational factors that influence the outcomes of the 
training program. For example, the interests, capabilities and 
job responsibilities of the trainees will influence the 
effectiveness of various training activities and the outcomes of 
a training program, in the evaluation described in part one, 
trainees were asked to comment on the training program and 
evaluate the pace of instruction, training materials, and 
instructional strategies. Also, pre and post-tests results were 
coded so that only the trainee would know his/her score. 

Other factors that will influence the design are the type of 
competencies being developed, the resources available at the 
training site, length of the training program, and the expertise 
available to conduct the evaluation. The qualify training 
program evaluation described earlier, focused on the development 
of cognitive skills that could be evaluated with a paper and 
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pencil test. Many training programs will include competencies 
that require performance tests. Most, if not all, of these 
competencies need to be evaluated with a performance test. In 
other words, the trainee must demonstrate the ability to do the 
task. 

Resources available at the training site will have an impact 
on the success of the training program. The impacts of these 
resources need to be evaluated and their adequacy determined. 
For example, in the training program described in the initial 
part of this paper, sufficient computer terminals were available 
so that no more than two trainees worked on a terminal. In the 
first program, the plant site provided the terminals and the 
result was five or six different types of terminals. The 
instructors and trainees suggested that only one type of terminal 
and one with paper output be used. 

Length of the training program may have an influence on the 
evaluation design. Longer training programs should include 
interim evaluation activities with feedback to trainees and 
trainers. 

The expertise available to conduct and interpret the 
evaluation activities must be considered. Scheduling a 
sophisticated performance test will not be appropriate if the 
personnel available do not know how to administer it. Also, the 
users of the evaluation results must be able to understand them 
and use the data in making decision. 
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Comparison and Control Groups . 

Usually one would like to have a comparison or control group 
to use in analyzing the evaluation data from a training program. 
A control group helps the evaluator isolate the impacts of the 
training program from the trainee changes that take place as the 
result of other events and experiences. 

Often it is difficult to obtain a true control group for an 
evaluation. However, one may be able to acquire a comparison 
group that is similar. For instance, another department with a 
similar mix of employees could be utilized, if the training 
program is going to be conducted for several departments, one or 
more of the departments waiting for the program could serve as a 
comparison group. When this type of design is used, pre and 
post-tests need to be administered. 

If control or comparison groups are not available, a time 
series design using job performance that could be used. For 
example, if an SFC training program is conducted to reduce the 
number of defective products produced, the impacts of this 
program could be evaluated by comparing the proportion of 
defective items produced before and after the training program. 
A time series design typically includes a baseline of data for 
several days or weeks prior to the training program. Performance 
levels after the training program are compared to this baseline. 
Often the baseline is available through control charts, pareto 
diagrams, or process capability data. Thus, it does not require 
much additional data collection. One caution needs to be 
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mentioned; however, the time series data will reflect all 
changes, events and training programs that take place during the 
time period studied. The impacts of a training program may be 
masked or enhanced by the impacts of other events. 

Whenever possible, a pre and post-test design should be 
used. This allows the evaluator to determine if the trainees,.^ 
have significantly improved their knowledge and performance 
levels. Administering a test or evaluation at the end of a 
training program gives a snapshot of the trainees' competency at 
that point in time. It does not indicate the changes that may 
have occurred. 

Use A Fol low-Up Study . 

One of the most powerful techniques available for evaluating 
training programs is a follow-up assessment of the trainee's 
performance. This can be done by collecting and analyzing data 
on individual trainee's performance or on the performance of a 
group. When possible, natural or typical data collected on 
processes and products should be used. Quality data car. be 
analyzed to determine if non-conforming parts or services 
decrease and if productivity increases. An effective Cost Of 
Quality (COQ) program should provide much useful data. Pareto 
and Cause-Effect diagrams can be reviewed to ascertain if common 
errors and problems have been reduced. 

A short-term follow-up completed in the three months after a 
training program will determine the immediate effects of a 
training program. It can also identify any constraints the 
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trainees are encountering as they try to use new competencies and 
procedures. A long-term follow-up study done nine to eighteen 
months after a training program will identify the long-term 
effects of a training program. 

A word of caution is appropriate at this point. The longer 
the time span between the training program and the follow-up 
evaluation, the greater the possibility that other factors may 
have had an impact on the trainees' performance. They may have 
participated in additional training programs, new equipment may 
have been installed, and/or suppliers may have improved their 
performance. Hence, it is important to review the trainee 
competencies assessed in the follow-up and determine the other 
factors that may influence performance. In general, it will be 
easier to track the impacts of a training program on specific 
technical competencies and knowledges than the impacts of more 
general training and educational programs. 

CIPP Design . 

One of the tools I have found very useful in designing 
evaluations is the CIPP Evaluation Model. The letters in CIPP 
represent Context, Input, Processes, and Products respectively. 
The model is based on the systems concept. (See Figure 4) 

The starting point in applying the CIPP model is to clearly 
identify the purpose of the training program, clear and 
descriptive training objectives are needed to accomplish this. 
These objectives will allow the evaluator to specify the trainee 
competencies that need to be measured (Products) . 
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After these competencies have been listed , the relevant 
input competencies of the trainees can be specified* The :input 
competencies will be the same as the output competencies , but may 
include some skills and knowledge that are used in the product, 
A pre-test is used to assess the input side* A post-test will be 
given to assess the product (s). 

The nature of the product and the inputs can be used to 
specify the types of process and context data needed* The 
processes will change the input to product* In this realm, the 
evaluator should focus on the adequacy and effectiveness of the 
training activities. The evaluator should consider observing 
training sessions and asking for feedback from instructors and 
trainees . 

Context is a more general variable* It includes the 
training facility , resources and environment. Again , the 
evaluator would want to use the product to help identify the 
context factors to be evaluated. For example , in the quality 
training program , one of the training objectives was to develop 
the ability to use MINITAB. This required access to MINITAB, a 
terminal and related instructional materials* Through the 
evaluation it was found that using one type of terminal made it 
easier to teach. Trainees also recommended that it was easier 
for them to use a terminal that had paper output. This change 
was made and the instructors found that it was simpler to teach 
and the trainees had fewer problems with the terminals. 
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Summary 

An effective evaluation is designed to measure attainment of 
the training program objectives, the evaluation must also 
provide data that is useful to the participants in the training 
process. Situational factors must be considered as the design is 
developed. Characteristics of the trainees, instructional 
setting and training materials have to be weighed as the design 
is developed. Some type of control or comparison group is needed 
to completely assess the impacts of a training program. Adding a 
follow-up study to an evaluation is an effective way to identify 
the impacts of the training program on trainees* on-the-job 
performance. Application of the CIPP evaluation model will 
provide direction for the evaluation design process. 
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Figure 4: CIPP Model Applied to a Training Program 
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APPENDIX D 
Statistics and Control Program 
Evaluation Form 
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Statistics and Control Program 
R valuation Fo r in 

0LUKCTI.ONS: Respond to each ol: the following statements based on your 
exporences in this traini nq program. Use the following 
responses. 

1 SI) Strongly Disagree 4 A Agree 

2 = I) - Disagree 5 = SA - Strongly Agree 

3 =- U Undecided 



Characteristics of the Training Program Responses 



SD D U A SA 



12 3 4 



rnir 394 395 

^ BEST COPY AVAILABLE 



1. This program was worthwhile 1 2 3 4 5 

2. I enjoyed using MINITAB 1 2 3 4 5 

3. MINITAB can be helpful in my work 1 2 3 4 5 

4. I gained a better understanding of statistics 12 3 4 5 

5. I will be able to use the contents of this 

training program 1 2 3 4 5 

5. I liked the opportunity to work with my own 

data 1 2 3 4 5 

7. There should be more opportunity to apply 
the contents to practical problems related 

to my work 1 2 3 4 5 

8. jl knew most of the content of this training 

program before we started the sessions . • . 1 2 3 4 5 

9- I learned a lot 1 2 3 4 5 

10. The instructors 1 explanations were clear . . 1 2 3 4 5 

11. The instructors effectively answered questions 12 3 4 5 

12. The instructors were able to relate the 
content of this tr<- ining program to practical 

problems 1 2 3 4 5 

13. The instructors were well prepared 1 2 3 4 5 

14. I plan to use some of the techniques presented 1 2 3 4 5 

15. Too many topics were included in the program 1 2 3 4 5 
S. Too few topics were included in the program . 1 2 3 4 5 

17. The visuals used were helpful 1 2 3 4 C J 



18. The handouts woro helpful 1 

19. How could the training program be improved? 

What did you like best about the '-raining program? 
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Work Based Learning Experience 
Introductio n 

There is no single approach to education which will satisfy 
the individual needs of each and every student. For an 
educational experience to hold the interest of the student, it 
should be perceived as being meaningful and attainable. Work 
based learning provides an alternative meai s of allowing students 
to internalize the value of their education experience. 

Purpose of this Module 

The purpose of this module is to describe selected forms of 
work base learning experiences which can be used as alternative 
educational models. 

Objectives 
The objectives of this module are to: 
1. Provide a general description of several forms of work based 
learning which include field trip, career education, simulation, 
job shadowing, mentoring, co-op, internship, and general work 
experience . 

Provide general information and example forms pertaining to 
the development, administration und evaluation of a work based 
learning experience program. 

Value of Work B ased Learning Experience 

Work based learning allows the student to develop work 
habits, skills and attitude, and provides exposure to the tools, 
equipment, materials, and work problems encountered in a real 
business and industry situation. Additionally, work based 

398 



31)9 



learning should pull bajic skills together to solve business and 
industry problems. 

General Objectives of a Work Based Learning Program 

The following general objectives have been identified for 
work based learning: 

1. Enhance the development of positive work habits, attitudes, 
and general employability skills for participating students. 

2. Provide participating students <«'ith a work based experiences 
requiring the practical application of oral communications, 
reasoning, problem solving, and basic skills. 

3. Provide exposure to up-to-date equipment and technology at a 
reasonable cost. 

4. Provide an understanding of the relationship between 
education and work. 

5. Identify work roles requirements and future educational 
opportunities appropriate for the abilities, desires, and 
interests of students. 

Forms of Work Base Learning 

Work based learning may take several forms which include; 
field trip, career education, simulation, job shadowing, 
mentoring, co-op, internship, and general work experience. 

General Description of Work Based Learning Experiences 

The following are brief descriptions of forms of work based 
learning programs which provide students with an opportunity to 
apply their formal educational experience in a work based 
experience. 
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Field Trip 

A field trip provides an opportunity to visit actual 
workplace sites, students can explore many types of work and 
identify the skills and training necessary for specific work. A 
field trip can increase the awareness of the role work plays 
within society and can be beneficial in determining a future 
career decision. Upon completion of the field trip, this 
experience should be evaluated by the teacher and students to see 
if the goals were met. 
Career Education 

Career education is another concept that exposes students to 
the skills and knowledge needed for exploring, understanding, and 
performing various life roles. Career education experiences 
often include linkages between earning and learning, it provides 
students with experiences designed to increase self -awareness, 
career awareness, and career decision-making. The Wisconsin 
Developmental Guidance Program Curriculum Guide published by the 
Department of Public Instruction contains educational activities 
with instruction to assist students in personal, social, career, 
and learning development. Components of this guide could provide 
ideas for work based learning experiences. 
Simulation 

Simulation provides students with short term experiences 
over a period of several weeks. The student gains awareness of 
the many tasks involved within a job. it provides a student with 
an opportunity to explore business, possible jobs, and enhance 
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their career development. 
Job Shadowing 

Job shadowing provides an opportunity to spend a day, or 
part of a day, on the job with a person who works in an 
occupation in which the student has an interest. The student 
trails the volunteer business professional around to understand 
the routine and requirements of the job. Information is provided 
by the business volunteer as to educational requirements, 
expected salary/benefits, and other options in that occupational 

field. 
Mentoring 

Mentoring is another form of a work based learning 
experience. The instructor obtains an experienced person in an 
occupational area to serve as a role model (or a career advisor) 
and assists in further developing student career exploration 
experiences. People serving as mentors are usually volunteers 
and provide a one-to-one relationship. 

Cooperative Occupational Education (Co-op). Inte rnship, and Work 
Experience 

Cooperative Occupational Education, Internships, and general 
work experience are directly worker based programs. Figure one 
shows a comparison between co-op, internship and general work 
experience. 
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Figure 1. Comparison of Three Plans Using the Work Environment 



Characteristics 



Student identifies 
career objective 
Related classroom 
instruction 
Established train- 
ing station and 
supervision by the 
school 
On-the-job 
training plan 
Paid employment 
Uses advisory 
committee 
VSO correlated 
with instruction 
Certified teacher 
in occupational 
area 

Planned home 
visitations 



Co-op | Internship 



| Work Exper i ence 



Yes 



Yes 



Yes 

Yes 
Yes 

Yes 

Yes 



Yes 



Sometimes yes | 



Yes |Not necessarily 



Not usually 



No 

Yes 
No 

No 

No 



No 



No 



No 



No 



No 

Not necessarily 
No 

No 
No 



No 



Source: Ralph E. Mason, "The Effective Use of Cooperative Work 
Experience . 11 Business Education Forum, Vol. 24, May 1970, 
p. 10. 
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Cooperative Occupational Education (Co-op) 
Co-ops use the occupational goals which are based on the 
student's career objective as the basis for this learning 
experience. The work situation could be a simulated laboratory 
that could be found in a classroom setting or a selected training 
station. The work situation is correlated with classroom 
instruction. The student usually receives pay as well as credit. 
There is supervision provided by the school instructor and work 
site host. This experience typically lasts one year. 

Internship 

In this situation, the student performs in an approved job 
situation. The student is usually paid and is given credit for 
this experience. In-school instruction is usually provided 
before the student begins the internship and may not necessarily 
be tied in directly with the job experience. This experience 
typically lasts one semester but can go two. In a secondary 
setting, this experience is referred to as work-study and on a 
postsecondary setting can be referred to as practicum or 
internship . 

Work Experience 
Work experience is a set of planned educational experiences 
which is design to enable students to acquire attitudes, skills, 
and knowledge of work through studying and observing. This type 
of experience is exploratory in nature and is usually 
accomplished in one semester or less. Usually there is no 
specific related class but rather work experience .is found woven 
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throughout a curriculum. 

Outcomes for Work Experience 

The basic outcome of work experience is a general 
understanding of the world of work and the development of 
acceptable work habits, attitudes, and personality traits. A 
student may learn some skills and information relevant to a 
particular occupation or job, but this is not the main purpose. 

The primary outcome is to help students develop those 
abilities, habits, and characteristics that are of value for any 
job. This experience provides the opportunity for career 
explorations and development of career interests. 

Training plans and agreements become part o£ these types oi. 
work based learning experiences. A training plan states the 
specific learning experiences which are to be offered based on 
the student's career goal. The purpose of the training plan is 
to organize learning and to correlate the classroom instruction 
with learning experiences at the actual work site. It is usually 
organized in a logical sequence and is based upon competencies 
needed for successful employment. 

A training agreement is a written agreement of the training 
commitment of the employer to the student. The purpose of a work 
based learning program is to give the student experience in a 
variety of tasks. The supervisor should sign this agreement 
befora the start of the experience to prevent any 
misunderstandings about the program. Cample training plans and 
agreement can be found in Appendices A and B respectively. 
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Evaluation 



The use of evaluation forms is one way of checking on the 
progress students are making in adjusting to and learning about 
work based learning experiences. Students gain insight into 
their progress which helps to identify their strengths and 
weaknesses. The evaluation form should be appropriate for the 
grade level of the student and the type of experience they have 
been given. Forms should be reviewed periodically to determine 
their appropriateness and the need for possible revision. As a 
general rule, the fewer the forms the better. Examples of 
student, instructor and business/ industry evaluations are 
provided in Appendices C through E. 
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APPENDIX A 
Training Plan 
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TRAINING PLAN 



Student Training Station 

Supervisor 

Tit le 

This training plan is an agreement between the employer/training 
sponsor and the coordinating teacher. The job supervisor trains 
the student-trainee for specific tasks on the job and provides as 
many different learning expeiiences as possible throughout the 
year. 

The school will provide the student with opportunities for 
training in the basic skills of the occupations and the technical 
information related to it. In order that a systematic plan can 
be followed which will provide well-rounded training for the 
position, a schedule of planned work experiences have been worked 
out. 

The student-trainee agrees to perform the work experiences 
assigned by the job supervisor according to the same company 
policies and regulations as apply to regular employees. The 
student-trainee also agrees to pursue the prescribed course of 
study and to take advantage of every opportunity to improve 
efficiency, knowledge, and personal traits so that he/she may 
enter any occupation as desirable employee. 

The parent or guardian's signatr. -3 indicates consent for the 
student- trainee to work and to study through the Work Experience 
Program. The job supervisor recognizes that the Work Experience 
Program prepares students with employability skills. 

Progress and advancement made by the student -trainee will be 
evaluated by the job supervisor each semester. The coordinator 
will cooperate with the training sponsor in an effort to plan 
classroom instruction in accordance with the work experience 

schedule. Training will cover the school year for a 

minimum of 15 hours per week. 



(Parent or Guardian) (job Supervisor) 



(Student-trainee) (Coordinating Teacher) 
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TRAINING PLAN 



Time Period_ Job Title 

Title 

Trainee Supervisor 

Job Task Rating Scale 

1. Can perform task satisfactorily without supervision. 

2. Can perform task with supervision - needs additional work. 

3. Cannot perform task satisfactorily. 

4. Has not experienced job task. 

Job Task Rating 
Job Tasks to Perform 1 2 3 4 
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APPENDIX B 
Training Agreement 
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TRAINING AGREEMENT 

High School 



Department of Public Instruction 
Wisconsin Department of Industry, Labor and Human Relations 

Name of student-trair ee: 

Name of Firm: 

Address of Firm: 

School : 



Occupational Title: 



The employer agrees to cooperate with the school (s) to offer a varied 
program. The employer agrees to pay the student-trainee a beginner's 

wage of $ per hour, furnish ratings of the student's on-the-job 

performance, and confer with the coordinating teacher periodically to 
determine in what way the student's learning in school and on the job 
might be strengthened. (If a sub-minimum wage is issued, the school 
must show acceptance by the Wisconsin Department of Industry, Labor 
and Human Relations and the U.S. Department of Labor when applicable.) 

The student-trainee agrees to perform to the best of their ability all 
duties assigned and conform to all rules and policies of their place 
of employment and the school. Student-trainees expect no special 
privileges, agrees to be on the job every day (barring illness), and 
will confer with their coordinating teacher regarding any problems 
encountered on the job. 

The parent or guardian agrees to cooperate with the school and 
employer to insure the best possible results from the student's 
training. 

The work schedule will be from to , Monday through 

Friday, from , 19 , to , 19 . In general, students 

should average a minimum of ten hours of work per week, not including 
Saturdays and school holidays. This average may spread over an entire 
school semester or year. The student should not exceed 40 hours per 
week of school and work experience (total hours in class on the job) 
Monday - Friday. 

THIS TRAINING AGREEMENT DOES NOT SUPPLEMENT THE WORK PERMIT. EMPLOYERS 
MUST BE ISSUED A WORK PERMIT FOR THE STUDENT BEFORE A STUDENT STARTS 
OCCUPATIONAL TRAINING. 



(Student-Trainee's Signature) (Employer's Signature) 

(Address) (Coordinating Teacher's Sign . ) 



(Parent's Signature) (Administrator's Signature) 



(Address) (Date) 



(Telephone Number) 
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THE STUDENT-TRAINEE WILL: 

1. Report promptly and engage in their assignment according to 
the training schedule. 

2. Cooperate with the agency supervisor, engage in the 
assignment as a training experience, observe etiquette, and 
keep safety rules. 

3. Notify the school and agency in advance when absence is 
unavoidable. 

4. Maintain satisfactory grades in all subjects in order to 
remain eligible for the program. 

5. Furnish the coordinating teacher with all necessary 
information and complete all necessary reports. 

6. Show honesty, punctuality, courtesy, a cooperative attitude, 
proper health and grooming habits, appropriate dress, and a 
willingness to learn. 

7. Remain with the employer during the training period. 

8. Abide by the rules and regulations of thr ~ perating 
agency . 

9. Keep all business information of the cooperating agency 
confidential. 

THE COOPERATING AGENCY WILL: 

1. Provide a training program, with varied experiences, which 
will contribute to the education of the student. 

2. Provide a job sponsor for the training of the student. 

3. Provide employment for the student during the agreed times. 

4. Adhere to all Federal and State regulations. 

5. Provide for the day-to-day safety of the student on-the-job 
THE PARENT OR GUARDIAN WILL: 

1. Be responsible for the conduct of the student while 
participating in the program. 

2. Be responsible for the method of transportation and for the 
student in delivery to and from their place of employment. 

3. Provide time for conferences with the coordinating teacher. 
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4. Become knowledgeable concerning the purposes and procedures 
of the training program. 

THE COORDINATING TEACHER WILL: 

1. Visit and assist the employer with establishing the training 
program . 

2. Observe the student on the job intermittently. 

3. Cooperate with the employer with the evaluation of the 
student. Final evaluation is the prerogative of the school. 

4. Make every attempt to solve problems that may arise from the 
cooperating agency, school, parent/ guardian, student, or 
community. 

5. Provide Meaningful in-school instruction related to the 
training activities of the student. 

6. Work with a local advisory committee to obtain assistance 
with the program. 

7. Provide general safety instructions to assist student. 

8. Cooperate with the employer in further safety training. 

The employer will comply with the provisions of Title IX of the 
Education Amendments oi! 1972, Section 504 of the Rehabilitation 
Act of 1973, and Title VI of the Civil Rights Act of 1964 by not 
discriminating on the basis of sex, handicap, race, color, or 
national origin in its treatment and assignment of students to 
jobs, hours of employment, levels of responsibility, and pay. 
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APPENDIX C 
Student's Program Evaluation Form 
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Student's Evaluation Form 
How has the experience you received been valuable to you? 



Could the training on-the-job be improved? 



If so, how? 



Was the instruction you received in school related to the work 
you were doing on the job? 



Can you make any suggestions which you feel would improve the 
classroom instruction? 



What parts of the classroom instruction did you find most 
beneficial? (discussions, reports, interviews, demonstrations, 
etc . ) ? 
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In general, how adequate was your work based learning 
experience for preparation in the following situations or life 
skills: (Please circle number 5, 4, 3, 2, or 1 for each 
statement . ) 

More Than Not 
Adequate Adequate Adequate 

Learning a specific job skills ... 5 4 3 2 1 

Learning a specific job knowledge . . 5 4 3 2 1 

Getting along with other workers . . 5 4 3 2 1 
Being able to talk to the boss about 

your problems 5 4 3 2 1 

Understanding the value and importance 

of work 5 4 3 2 1 

Solving problems on the job 5 4 3 2 1 

Understanding the U.S. and world 
economic systems (supply-demand, 

inflation, recession, etc.) 5 4 3 2 1 

Managing career responsibilities with 

home and family responsibilities. . . 5 4 3 2 1 

Applying basic skill preparation in 
math, reading, writing, spelling, 
speaking, listening to every day 

problems 5 4 3 2 1 

Please make any suggestions you have for improving the program. 
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APPENDIX D 
Instructor's Student Evaluation Form 
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Evaluation by Instructor 
Student's Name 

Please rate the following statements by placing an "X" in the 
appropriate space. 

Excellent = A; Above Average = B; Average = C; Poor = D. An "F" 
shows failure to meet minimum standards. Not Apjjlicable=N/A. 

Characteristic A B C D F N/A 

Demonstrates willingness to learn 

Exhibits acceptable personal growing 

Reports to work on time_ 

Applies self to the job 

Adapts to new situations 

Initiates work without supervision_ 

Completes the job satisfactorily 

Possesses emotional stability 

Shows self confidence 

Follows directions 

Accepts criticism 

Exhibits ability to work with others_ 

Works safely 

Works above the beyond demands 

Note: It is suggested that students be evaluated every f 
weeks so they will have an opportunity to improve, it may be 
good procedure to ask students to first evaluate themselves, it 
is very important to sit down with students and discuss the 
evaluation once it has been completed. Remember, positive 
feedback and strategies for improving are an important part of 
the evaluation. 
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APPENDIX E 

Business 's/ Industries' Program Evaluation Form 
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Business 's and Industry's Student Evaluation 



Do you feel the student was placed in a program that best 
suited their interests and abilities? 



Do you feel the student lacked preparation for the 
on-the-job training position? 



Please rate the student in the following areas: (Circle the 
appropriate responses) 

Well Adequately Not at all 
Prepared Prepared Prepared 
LA} (2) M) 

3. Occupational knowledge . 5 

4. Occupational skills . . 5 

5. Work attitude 5 

6. Work ^uality 5 

7. Work quantity (amount 
produced) 5 

8. Was the teacher/coordinator's supervision of the student 
adequate and helpful? 
Explain. 



4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 



Was there cooperative effort on the part of the teacher/ 
coordinator and training supervisor in developing individual 
related classroom instruction to benefit the student's 
on-the-job training? if yes , was it effective? 
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In the following areas of basic skills, how well prepared was 
this student? (Circle the appropriate responses) 

Well Adequately Not at all Not 

Prepared Prepared Prepared Observed 

(5) _i4) m m (ju cpj__ 



10. 


Reading .... 


5 


4 


3 


2 


1 


0 


11. 


Spelling . . . 


5 


4 


3 


2 


1 


0 


12. 


Computer Literacy 


5 


4 

• 


3 


2 


1 


0 


13. 


"ath 


5 


4 


3 


2 


1 


0 


14. 


Science .... 


5 


4 


3 


2 


1 


0 


15. 


Writing .... 


5 


4 


3 


2 


1 


0 


16. 


Speaking . . . 


5 


4 


3 


2 


1 


0 


17. 


Listening . . . 


5 


4 


3 


2 


1 


0 


18. 


Problem Solving 


5 


4 


3 


2 


1 


0 


19. 


To what degree did the 
and work habits? 


student develop 


the 


proper 


attitudes 



Please rate the student on each of the characteristics listed 
below. (Circle the appropriate responses) 



Has Made Needs 
Grown Improve- Improve- 
ment m^nt 
_0) C4J CD m OJ 



Not 
Observed 

CO) 



20. Use of tools and 
equipment . . . 

21. Selection and care of 
space, materials, 
and supplies . . 5 

22. Ability to work with 
co-workers ... 5 

23. Accepting advice and 
supervision . . 5 
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Has Made Needs Not 

Grown Improve- Improve- Observed 
ment ment 

is) cd m m ui (q) 

24. Working in a leader- 

ship role when 

applicable ... 5 4 3 21 0 

25. Understanding the value 
and importance of 

work 5 4 3 2 1 0 

26. Prompt and dependable 

in attendance .54321 0 

27. Willing to accept and 

perform tasks .54 3 2 1 o 

28. Ability to plan and 

direct own work 5 4 3 2 1 o 

29. Ability to make sound 

decisions ... 5 4 3 2 l o 

30. Shows consideration 
and kindness towards 

others ....,54 3 2 1 o 

31. Neat, clean, and 
appropriate 

appearance ... 5 4 3 2 l o 

32. Flexibility to accept 

change 5 4 3 2 1 0 

33. Ability to learn new 

job tasks ...54 3 2 1 0 

34. Please make any comments or suggestions that would improve 
the program . 
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Developing Partnerships Between Education 
and Business/ Industry 

Introduction 

In today's intensely competitive global economy, 
partnerships between educational institutions and the 
business/ industrial sector can yield many important benefits to 
both parties. The educational process is enhanced by involving 
students in real-world problems and by gaining access to state of 
the art equipment and leading edge technical expertise. Industry 
gains access to specialized technical expertise, research 
capabilities and future employees. 

When attempting to establish such partnerships, the primary 
function of both parties to the partnership must be kept firmly 
in mind. Educational institutions have a primary mission to 
teach (research and public service are secondary goals) and 
inO'istry is in business to produce the best products at the 
lowest price. 

From an educational point of view, the real value of 
participating in a partnerships with businesses/industries lies 
in their potential to improve the educational process through 
faculty development and by allowing students to participate in 
realistic experiences. If these considerations can be satisfied, 
such partnerships are of immense value. 

Purpose 

The purpose of this module is to provide on overview of the 
considerations and benefits associated with developing and 
maintaining partnerships involving educational institution and 
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representatives from businesses and industries. 

Objectives 
The objectives of this module are to; 

1. Describe the considerations associated with developing 
partnerships involving educational institutions and 
representatives from businesses/industries. 

2. Provide information concerning the value to both parties for 
maintaining a successful partnership. 

3. Describe the benefits of continued, long-term interaction. 

Developing Partnerships 
Developing connections with the outside world can be a 
daunting task for educational institutions who have little prior 
experience with outreach. Appendix A graphically portrays the 
benefits of educational and business/industrial interaction and 
the relationships that need to be developed for a successful 
partnership. 
Culture 

Partnerships can be developed in many ways. To be 
successful, however, both parties to the partnership must view 
the potential interaction as a desirable and profitable venture, 
and both partners must be willing to commit resources to the 
venture. In short, a culture that values and promotes 
interaction must exist within both of the participating 
organizations. If it does not already exist, developing this 
intra organizational culture will be the first, and perhaps the 
most difficult task to be addressed. Educational institutions 
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may find that developing this culture is a slow process, as many 
faculty feel their primary task is to teach, not interact with 
industry. With patience and perseverance, the value of 
interaction to the instructional process can be amply 
demonstrated for most faculty. Once developed, the appropriate 
culture should be reasonably easy to maintain. 
Building Bridges 

In parallel with developing an intra organisational culture 
that favors interaction, bridges to the outside world must be 
developed. That is, the educational institution's presence, 
capabilities, and willingness to interact must be made known to 
industry. One of the best ways to accomplish this is with the 
assistance of industrialists who see the advantages of 
interaction. 

The University of Wisconsin-Stout has a formal mechanism 
called the Business and Industrial Professor Program. In this 
program, Industrialists who have retired or who are about to 
retire are formally engaged to assist the institution by having 
them sign a formal contract, usually at no salary. The Business 
and Industrial Professor enjoys all of the rights and privileges 
of a tenured faculty member (including "hunting" for parking 
spots) . They may volunteer or be engaged as staff to teach 
class, conduct research, assist outreach projects or perform 
other duties. Their greatest value lies in their contacts with 
the industrial community and their ability to connect the 
institution with the right "people" in industry. 
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Operational Policy 

Successful partnerships are best developed and sustained 
when there are clear guidelines for interaction, preferably a 
formal institutional policy that prescribes how certain issues 
are to be resolved, such as: indirect costs, patent ownership, 
publication, confidentiality, etc. Another issue of great 
interest to industry is meeting deadlines — can the project be 
completed on time at the stated cost? Education on the other 
hand, has greater concern for enhancing the instructional 
process, rathe': than meeting the deadline. Given time and 
commitment, these issues can be resolved. 
Mechanisms 

Making connections with potential partners can occur in a 
variety of ways. Some mechanisms for making connections or 
identifying potential partners are as follows: 

1. Networking . Institutional faculty attend workshops, 
seminars, conventions, professional society meetings, etc., and 
dv/elop a network of connections. 

2. Advisory committees . Membership on advisory committees can 
be thoroughly introduced to the strengths of the institution and 
the potential for interaction. A very effective mechanism. 

3. Industry Days . The institution hosts business/ industry 
through the invitation of Chambers of Commerce or Industrial 
Development Corporation. The focus is to introduce the 
participants to the educational institution's faculty and 
resources. 

428 

4 :ii 



4. Campus Visits. Representatives of individual companies are 
hosted by unit administrators and provided an overview of the 
institution, its capabilities and interest. This is a time 
consuming process, but extremely effective. 

5. Speaking Engagements . Representatives of the institution 
should promote it's strengths, goals and successes at every 
opportunity; speaking at conferences, workshops, economic 
development meetings, industry groups; or other meetings. 

6. Awards Conferences . Recognition of industry and/or 
educational leaders for their efforts to promote interaction, the 
profession or other desirable ends. This is an opportunity to 
showcase the institution's more successful projects. 

Principal Contract 

While many people may be involved x n identifying potential 
opportunities for interaction, the process of formally 
establishing connections should be concentrated in one office. 
There will be fewer opportunities fcr misunderstandings or 
misinformation with one source of information, one central point 
of contact is desirable for everyone. Where the office or 
contact individual is located is not overly important, the main 
contact person must have an intimate knowledge of the 
institution's capabilities and staff expertise. Much will depend 
upon the contact person being able to conceptualize many 
different ways in which a technical problem can be addressed and 
being able to identify staff expertise from across the campus, 
not just in his/her school or department. 

429 



4:10 



Again, it is important that there be one set of rules for 
interaction with outside agencies and that everyone is asked to 
play by the same rules — i.e. no favors for any group! The office 
that arranges interaction should have the authority to speak on 
behalf of the university and make commitments relative to some of 
the issues cited above. This will make it easier to arrange 
potential interaction . 

Interaction/Projects 
A part of the process of promoting interaction r rill be some 
institutional self analysis to identify strengths and the areas 
where it wishes to pursue projects. These will need to be 
identified along with faculty consultants/ experts for de :elopment 
of promotional material. Potential projects would then ^ sought 
from among t!*e several companies where a relationship has been 
formed. 

Initiating a successful project will require: (a) 
demonstration of requisite technical expertise and potential to 
successfully conclude a project, (b) clear identification of the 
project parameters, (c) expected project outcomes, (d) time frame 
for completion, (e) specific goals to be achieved, (f) 
comprehensive budget, and (g) confidentiality and publication 
considerations . 

Continuing a healthy partnership/relationship will depend 
upon how well the project goals were satisfied, the value the 
company perceives it received from the project, and any benefit 
that may accrue from a continuation of the relationship. In many 
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cases, projects may be successfully completed with complete 
sponsor satisfaction and no expectation of continuation. In 
other cases, the project may yield more questi ns than answers 
and suggest a continuation along specific lines. 

Maintaining A Partnership 

Successful completion of a project does not mean an end to 
the partnership. It will be important for the company and the 
institution to stay in contact even when no formal activity is 
underway. Maintaining this relationship will require some effort 
from the institution's leadership and involve such things as: 
(a) occasional telephone calls, (b) letters or other 
correspondence, (c) invitations to assist the institution in 
other projects or in instructional programs, and (d) requests for 
capital support. 

Both the institution and industry have much to gain from 
continued interaction. The institution may desire added capital 
equipment, advice or guidance on new technology, assistance in 
specialized instruction, connections to other industries, 
programmatic assistance, or other assistance. Industry may need 
specific advice on a process, aid in securing students to 
participate in a cooperative educational experience (Co-op) or 
internship, names of experienced graduates, speakers for 
workshops, assistance in organizing instruction, and related 
work. By continuing an informal relationship, trust and respect 
can be developed between the two organizations that will yield 
long-term benefits in the form of mutual support and/or 
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additional funded projects. 

Benefits of Continued Interaction 

The benefits of continued interaction between education and 
industry can be as varied as the number of established 
partnerships. While we have already cited numerous specific 
benefits for both education and business/ industry , one of the 
prime benefits is the potential for networking and developing 
totally new connections. 

For example, an institution may successfully complete a 
materials-related project for company A and discover later that 
the company has great needs in the area of statistical process 
control at the subsidiary plant in another state. Using its 
expertise in SPC, the institution may initiate instructional 
programs with the subsidiary. While conducting these 
instructional programs, contact may be initiated with the 
subsidiary of a totally different company that results in 
securing assistance in program development from that company. 

Benefits may be slow to emerge, it may take several years 
for the right project to 'jell.' This may entail getting 
involved with several divisions of the company or working with 
the same division on several project proposals before the right 
one emerges. As a case in point, the University of Wisconsin- 
Stout has been involved with a major defense contractor for at 
least eight (8) years with no concrete project work forthcoming. 
A number of projects involving both low-technology (trailing 
edge) and high-technology (leading edge) components were proposed 
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with no results. In the meantime, the defense contractor has 
supplied numerous pieces of obsolete equipment (obsolete to the 
company but not to the institution) , corporate staff have served 
on Several advisory boards and they have committed numerous 
technical staff to assist the institution. Within the last 
several months, the company and the institution are again 
discussing a project that is likely to result in a new production 
facility and product line being established adjacent to the 
university. The point is that long-term relationships are 
mutually beneficial. 

Summary 

One never knows what will develop from a contact. Whether 
it is casual or carefully planned, there appears to be a great 
deal of serendipity involved in making connections. A casual 
contact with someone on an airplane may yield a long-term project 
and relationship, while a carefully laid out program with a 
hometown company may never bear fruit. 

Developing a productive and successful outreach program 
requires proactive and committed individuals from both sides of 
the partnership who believe in the importance and value of 
interaction. These are the individuals who seize every 
opportunity to establish a connection; who promote dialogue 
between potential collaborators; who rise early and stay up late 
to write proposals and promote deals; who speak freely, and at 
every opportunity, on the value of interaction; who continue to 
prod, push and cajole. Without these champions, little is 
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accomplished. With them endless horizons ot opportunity become 
available. 




APPENDIX A 



Model for Developing Education and 
Business/ industry Partnerships 
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DEVELOPING EDUCATION 
AND BUSINESS PARTNERSHIPS 



Business/Industry 



Needs 



Ideas 

Technical Support 
Training 

Short Term Staff 
New Technology 
§ Research Facilities 

Resources 



Real-Time Problems 
Real Laboratory 
Realistic Problems 
Co-op/Intern Stations 
Leading Edge 
Technology 



Interface 

Culture... 

A focus on outreach. 



Bridges... 

Building mechanisms for interaction. 
Operational Policy... 

Established and consistent methods. 
Patent Policy... 

Formal statement. 



Connections... 

Methods for establishing connections, 



University/College 



Resources 



Faculty/Staff/Students 
Research Facilities 
Technical Labs 
Management Expertise 
Technical Expertise 
Business Expertise 



Needs 



Staff Development 
New Skills 
New Technology 
Co-op/Intern Stations 
Real-Time Projects 
Realistic Projects 



CHAMPIONS ! 



* 4.J7 
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